From d382089a973d3b08a836e609917cf7bff251c564 Mon Sep 17 00:00:00 2001 From: Jean-Baptiste THERY Date: Sun, 1 Nov 2026 19:17:53 +0700 Subject: [PATCH 1/5] feat(core): calibrate abstention per index and report actionable index health - measure each Transformers index's noise floor at ingest from 24 off-topic probe questions, store it in the manifest, and measure it once per process for older indexes - treat a passage as a strong vector match only when it beats the noise floor, and answer nothing when the first five semantic results are all weak - flag mxbai-embed-xsmall passages as weak below 30% of the question's words instead of 50%, which flagged a third of the correct answers on a code repository - select `_score` in term-frequency queries so LanceDB stops printing a deprecation warning on every search - throw CONFIG_MIGRATION_REQUIRED instead of an internal error for configs with retired fields - have doctor ask for `rgr storage optimize` on fragmented indexes and for `rgr ingest` when a semantic index has no stored noise floor - document the noise floor, weak answers, the migration notes, and the new error code --- docs/agent-integration.md | 3 +- docs/api-reference.md | 5 +- docs/cli-reference.md | 9 +- docs/configuration.md | 17 +- docs/migration.md | 16 ++ docs/troubleshooting.md | 4 +- llms.txt | 5 +- packages/ragmir-core/skills/ragmir/SKILL.md | 2 + .../src/abstention-calibration.test.ts | 163 ++++++++++++++++++ .../ragmir-core/src/abstention-calibration.ts | 149 ++++++++++++++++ .../ragmir-core/src/config-migration.test.ts | 12 ++ packages/ragmir-core/src/config.ts | 10 +- packages/ragmir-core/src/doctor.ts | 22 ++- packages/ragmir-core/src/errors.ts | 1 + packages/ragmir-core/src/ingest.ts | 37 ++++ packages/ragmir-core/src/query.ts | 32 +++- packages/ragmir-core/src/ranking.test.ts | 77 ++++++++- packages/ragmir-core/src/ranking.ts | 75 ++++++-- .../src/storage-maintenance.test.ts | 18 ++ .../ragmir-core/src/storage-maintenance.ts | 18 ++ packages/ragmir-core/src/types.ts | 15 ++ tests/public-api-consumer/consumer.ts | 2 + 22 files changed, 657 insertions(+), 35 deletions(-) create mode 100644 packages/ragmir-core/src/abstention-calibration.test.ts create mode 100644 packages/ragmir-core/src/abstention-calibration.ts create mode 100644 packages/ragmir-core/src/storage-maintenance.test.ts diff --git a/docs/agent-integration.md b/docs/agent-integration.md index 1dc1298..1921a34 100644 --- a/docs/agent-integration.md +++ b/docs/agent-integration.md @@ -134,7 +134,8 @@ questions appears once. Within one MCP session, passages already returned come b Results flag two conditions. `sourceStatus: "modified"` or `"missing"` means the cited file changed after indexing; with `mcpAutoIngest: true`, the server then starts one background incremental ingest (at most once a minute). `weakEvidence: true` means the passage holds few of the question's -distinctive words and has no strong semantic match. +distinctive words and has no strong semantic match. An empty answer means the base holds no +evidence for the question: say so instead of answering from the nearest passages. The tools accept common argument spellings (`top_k`, `radius`, `max_bytes`, `paths`, `q`) and clamp out-of-range numbers instead of failing; the response notes each adjustment and lists ignored diff --git a/docs/api-reference.md b/docs/api-reference.md index dbb31a8..ecc36ee 100644 --- a/docs/api-reference.md +++ b/docs/api-reference.md @@ -71,7 +71,8 @@ It identifies indexed evidence, not the current file on disk. Each result whose disappeared since indexing also carries `sourceStatus: "modified"` or `"missing"`, from one `stat` per cited file; `doctor` checks the whole corpus by size and modification time, and `audit` by checksum. `weakEvidence: true` marks a passage with few of the question's distinctive words and -no strong vector match. Compact results retain these fields. +no strong vector match; a Transformers search whose first five results are all weak returns an +empty list instead. Compact results retain these fields. For XLSX tables, full results attach the header as a separate `context` passage with its own cell citation, including at `contextRadius: 0`. Expansion also includes it; compact search requires @@ -119,6 +120,8 @@ it excludes paths, source text, and queries. No telemetry is transmitted. Operations accepting `OperationOptions` support `signal` and `timeoutMs`. The library returns values or throws errors; it does not write to stdout/stderr. Use `isRagmirError(error)` to inspect structured codes such as `TIMEOUT`, `OVERLOADED`, `INDEX_BUSY`, `INDEX_UNAVAILABLE`, and `CLIENT_CLOSED`. +A config that still holds retired fields throws `CONFIG_MIGRATION_REQUIRED`: run `rgr upgrade` to +back it up, migrate it, and rebuild the index. Per-project process-local queues bound search, embeddings, and ingestion. A private writer lock serializes index writers across local OS processes. Readers keep generation leases through their diff --git a/docs/cli-reference.md b/docs/cli-reference.md index 6c4da57..485a491 100644 --- a/docs/cli-reference.md +++ b/docs/cli-reference.md @@ -46,8 +46,9 @@ rgr search "migration" --exact-vector-search Text output marks a result whose file changed or disappeared after indexing (`source modified since indexing`) and a result with thin support (`weak evidence`). JSON results -carry the same information as `sourceStatus` and `weakEvidence`. `--scope` applies a named scope -from the config; `--include-path` accepts backslashes on every platform. +carry the same information as `sourceStatus` and `weakEvidence`. A semantic search whose first five +results are all weak prints no result: the base holds no evidence for the question. `--scope` +applies a named scope from the config; `--include-path` accepts backslashes on every platform. `sources add` accepts paths, globs, and `!` exclusions. Search accepts `--top-k`, `--include-path`, `--exclude-path`, and repeatable `--context-path`. Search accepts @@ -202,7 +203,9 @@ doctor and `rgr audit` label their O(corpus) cost in text and JSON output. A mis manifest always yields `ready=false`, including legacy tables that predate manifest activation. `readiness.lexicalIndexCurrent` is `false` for an index built under an older keyword policy, such as one without accent-folded terms or path and identifier terms. Search keeps using that index; the -next `rgr ingest` or `rgr upgrade` stages and validates a rebuild before activating it. +next `rgr ingest` or `rgr upgrade` stages and validates a rebuild before activating it. Its next +steps also ask for `rgr storage optimize` when the index is split into many small fragments, and for +`rgr ingest` when a semantic index has no stored noise floor yet. `rgr ingest --rebuild` writes batches into an isolated LanceDB generation. The existing index stays active until the new table and manifest pass row-count, checksum, and duplicate-ID validation. The diff --git a/docs/configuration.md b/docs/configuration.md index bbc4ead..64f6692 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -149,11 +149,20 @@ distinctive word counts more than a common one: `mxbai-embed-xsmall-v1` uses 1.1. Other models keep the previous rule: any related query word, or a distance up to 1.1. +Every Transformers index also measures its own noise floor: `rgr ingest` embeds 24 fixed off-topic +probe questions (sports, cooking, geography, nonsense, in English and French) and stores the +distance of the nearest passage any of them reaches. A model packs texts into its own band and a +corpus shifts it, so only a passage nearer than every off-topic probe counts as a strong vector +match, and a question with no known word needs one. An index built before this release measures +the floor once per process on its first search; the next `rgr ingest` stores it. + Each result whose passage holds few of the question's distinctive words and has no strong vector -match carries `weakEvidence: true`. It is still returned; agents should verify it first. On a -French and English golden set over a Java and TypeScript monorepo, these rules kept every -answerable question's source, and with `mxbai-embed-xsmall-v1` they rejected 16 of 30 unrelated -questions instead of 3. Evaluate your own corpus with `rgr evaluate`. +match carries `weakEvidence: true`. When the first five results of a Transformers search are all +weak, the answer holds no real evidence and search returns nothing. `local-hash` search keeps such +results. On a Java and TypeScript monorepo, a product documentation base, and this repository +(260 answerable questions), these rules kept every answer in place while refusing 58 of 90 +off-topic and 24 of 90 near-domain unanswerable questions, against 41 and 1 before, and flagged +14 correct answers as weak instead of 43. Evaluate your own corpus with `rgr evaluate`. Changing an embedding provider, model, revision, digest, or chunking field requires `rgr ingest --rebuild`. Revision and artifact digest participate in the index, vector-index, and diff --git a/docs/migration.md b/docs/migration.md index beeb633..b8e5199 100644 --- a/docs/migration.md +++ b/docs/migration.md @@ -3,6 +3,22 @@ This is a breaking surface change. Ragmir now concentrates on local indexing, search, exact citations, OCR, and integration with developer agents. +## Index noise floor and weak answers (6.4) + +Existing indexes stay searchable after the update; nothing needs deleting. + +- **Ranking policy version 8.** Transformers search caps strong vector evidence with each index's + noise floor and answers nothing when its first five results are all `weakEvidence`. + `mxbai-embed-xsmall-v1` flags a passage as weak below 30% of the question's IDF-weighted words + instead of 50%. Rerun `rgr evaluate`: stored quality reports no longer verify the active policy. +- **Noise floor.** `rgr ingest` stores the floor in the index manifest. An older index measures it + once per process on its first search until the next `rgr ingest`. +- **Typed migration error.** A config with retired fields now throws `CONFIG_MIGRATION_REQUIRED` + instead of an internal error. Run `rgr upgrade`. +- **Fragmented indexes.** `rgr doctor` asks for `rgr storage optimize` when an index is split into + many small fragments, which slowed searches up to five times on indexes built before automatic + compaction. + ## Canonical paths, live freshness, and calibrated evidence (6.2) Existing indexes stay searchable after the update; nothing needs deleting. diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md index 992a91e..1ba2419 100644 --- a/docs/troubleshooting.md +++ b/docs/troubleshooting.md @@ -94,7 +94,9 @@ need explicitly configured local extractors. ## Search is weak Results marked `weakEvidence` hold few of the question's distinctive words and have no strong -semantic match: rephrase with the project's own terms or narrow the search with a `scope`. When one +semantic match: rephrase with the project's own terms or narrow the search with a `scope`. A +semantic search returns nothing when its first five results are all weak; a rephrased question +with the project's vocabulary usually finds the evidence if it exists. When one export or log dominates results, `rgr audit` reports index concentration; exclude it, lower it with `sourceWeights`, or cap it with `maxChunksPerDataFile`. diff --git a/llms.txt b/llms.txt index 8ee63e5..99e8c38 100644 --- a/llms.txt +++ b/llms.txt @@ -31,8 +31,9 @@ the query. It owns reasoning, generation, and actions, with the model chosen by session as short stubs. Instructions name the base; unrelated workspaces get a scope notice, or a refusal with `mcpAllowedRoots`. Register Ragmir per project, not for every project. - Results carry `sourceStatus` when the cited file changed after indexing and `weakEvidence` when - support is thin. `rgr doctor` checks freshness on every run; `rgr hooks install` refreshes the - index after Git operations. + support is thin. An empty semantic answer means the base holds no evidence for the question. + `rgr doctor` checks freshness on every run; `rgr hooks install` refreshes the index after Git + operations. - Agents search, expand selected evidence, and refine queries. Ragmir does not run the agent loop. - Present four workflow examples: build a feature, diagnose an incident, and plan a migration with Claude Code or Codex using cited project evidence; then make a confidential architecture decision diff --git a/packages/ragmir-core/skills/ragmir/SKILL.md b/packages/ragmir-core/skills/ragmir/SKILL.md index 69705ce..fad0434 100644 --- a/packages/ragmir-core/skills/ragmir/SKILL.md +++ b/packages/ragmir-core/skills/ragmir/SKILL.md @@ -30,6 +30,8 @@ Old redaction/privacy settings require this explicit migration. Source text is n 3. Expand one selected citation with `ragmir_expand`. Use `compact: false` only when full search passages are needed. 4. Iterate when evidence is incomplete. Separate facts backed by citations from inference. + Verify a `weakEvidence` passage before relying on it. An empty answer means the base holds no + evidence: rephrase once with the project's own terms, then say so instead of guessing. | Need | CLI or MCP | | --- | --- | diff --git a/packages/ragmir-core/src/abstention-calibration.test.ts b/packages/ragmir-core/src/abstention-calibration.test.ts new file mode 100644 index 0000000..567126c --- /dev/null +++ b/packages/ragmir-core/src/abstention-calibration.test.ts @@ -0,0 +1,163 @@ +import type { Table } from "@lancedb/lancedb" +import { describe, expect, it } from "vitest" +import { + ABSTENTION_PROBES, + indexNoiseFloor, + manifestNoiseFloor, + measureAbstentionCalibration, + noiseFloorAdvice, +} from "./abstention-calibration.js" +import type { Config, IndexManifest } from "./types.js" + +const semanticConfig = { + embeddingProvider: "transformers", + embeddingModel: "intfloat/multilingual-e5-base", +} as Config + +/** A table whose exact nearest distance for probe vector [i] is distances[i]. */ +function tableWithNearest(distances: Array): Table { + return { + vectorSearch: (vector: number[]) => { + const query = { + bypassVectorIndex: () => query, + select: () => query, + limit: () => query, + toArray: async () => { + const distance = distances[vector[0] ?? 0] + return distance === undefined ? [] : [{ id: "row", _distance: distance }] + }, + } + return query + }, + } as unknown as Table +} + +const embedByIndex = async (texts: string[]) => texts.map((_, index) => [index]) + +describe("abstention calibration", () => { + it("should keep the nearest off-topic distance as the index noise floor", async () => { + const distances = ABSTENTION_PROBES.map((_, index) => 0.5 - index * 0.001) + distances[3] = 0.31 + distances[7] = undefined + + const calibration = await measureAbstentionCalibration( + tableWithNearest(distances), + semanticConfig, + embedByIndex, + ) + + expect(calibration).toEqual({ + schemaVersion: 1, + embeddingModel: "intfloat/multilingual-e5-base", + probeCount: ABSTENTION_PROBES.length - 1, + noiseFloorDistance: 0.31, + }) + }) + + it("should skip local-hash indexes and empty tables", async () => { + const localHash = { ...semanticConfig, embeddingProvider: "local-hash" } as Config + expect( + await measureAbstentionCalibration(tableWithNearest([0.2]), localHash, embedByIndex), + ).toBeNull() + expect( + await measureAbstentionCalibration(tableWithNearest([]), semanticConfig, embedByIndex), + ).toBeNull() + }) + + it("should read a stored floor only for the configured model", () => { + const manifest = { + abstentionCalibration: { + schemaVersion: 1, + embeddingModel: "intfloat/multilingual-e5-base", + probeCount: 24, + noiseFloorDistance: 0.31, + }, + } as IndexManifest + + expect(manifestNoiseFloor(manifest, semanticConfig)).toBe(0.31) + expect( + manifestNoiseFloor(manifest, { ...semanticConfig, embeddingModel: "other/model" } as Config), + ).toBeNull() + expect(manifestNoiseFloor(null, semanticConfig)).toBeNull() + }) + + it("should advise an ingest only for a semantic index without a stored floor", () => { + const run = (args: string[]) => `rgr ${args.join(" ")}` + const legacy = { chunkCount: 120 } as IndexManifest + const stored = { + chunkCount: 120, + abstentionCalibration: { + schemaVersion: 1, + embeddingModel: "intfloat/multilingual-e5-base", + probeCount: 24, + noiseFloorDistance: 0.31, + }, + } as IndexManifest + const localHash = { ...semanticConfig, embeddingProvider: "local-hash" } as Config + + expect(noiseFloorAdvice(legacy, semanticConfig, run)).toContain("Run `rgr ingest` to store it.") + expect(noiseFloorAdvice(stored, semanticConfig, run)).toBeNull() + expect(noiseFloorAdvice(legacy, localHash, run)).toBeNull() + expect(noiseFloorAdvice({ chunkCount: 0 } as IndexManifest, semanticConfig, run)).toBeNull() + }) + + it("should measure a legacy index once per generation and reuse a stored floor", async () => { + const distances = ABSTENTION_PROBES.map(() => 0.42) + let embedCalls = 0 + const countingEmbed = async (texts: string[]) => { + embedCalls += 1 + return embedByIndex(texts) + } + const legacy = { chunkCount: 12 } as IndexManifest + const table = tableWithNearest(distances) + + expect( + await indexNoiseFloor(table, semanticConfig, legacy, "generation-a", countingEmbed), + ).toBe(0.42) + expect( + await indexNoiseFloor(table, semanticConfig, legacy, "generation-a", countingEmbed), + ).toBe(0.42) + expect(embedCalls).toBe(1) + + const stored = { + chunkCount: 12, + abstentionCalibration: { + schemaVersion: 1, + embeddingModel: "intfloat/multilingual-e5-base", + probeCount: 24, + noiseFloorDistance: 0.3, + }, + } as IndexManifest + expect( + await indexNoiseFloor(table, semanticConfig, stored, "generation-b", countingEmbed), + ).toBe(0.3) + expect(embedCalls).toBe(1) + }) + + it("should keep the policy distances when the measurement fails or does not apply", async () => { + const failing = async () => { + throw new Error("model unavailable") + } + const legacy = { chunkCount: 12 } as IndexManifest + const localHash = { ...semanticConfig, embeddingProvider: "local-hash" } as Config + + expect( + await indexNoiseFloor( + tableWithNearest([0.2]), + semanticConfig, + legacy, + "generation-c", + failing, + ), + ).toBeNull() + expect( + await indexNoiseFloor( + tableWithNearest([0.2]), + localHash, + legacy, + "generation-d", + embedByIndex, + ), + ).toBeNull() + }) +}) diff --git a/packages/ragmir-core/src/abstention-calibration.ts b/packages/ragmir-core/src/abstention-calibration.ts new file mode 100644 index 0000000..1b4fa07 --- /dev/null +++ b/packages/ragmir-core/src/abstention-calibration.ts @@ -0,0 +1,149 @@ +import type { Table } from "@lancedb/lancedb" +import type { AbstentionCalibration, Config, IndexManifest } from "./types.js" + +/** + * Off-topic questions in English, French, and nonsense, chosen to share no subject with a software + * project. Their nearest passages mark where unrelated questions land in an index, whatever the + * embedding model. Changing the list changes every calibration, so keep it stable. + */ +export const ABSTENTION_PROBES: readonly string[] = [ + "Who won the Super Bowl in 1998?", + "How long should I boil an egg?", + "What is the population of Canada?", + "Best beaches in Portugal for surfing", + "How to knit a scarf for beginners", + "What time is sunset in Tokyo today?", + "Who were the members of the Beatles?", + "How do volcanoes form?", + "Recommend a good fantasy novel", + "What is the exchange rate of the euro to the yen?", + "Quelle est la capitale de l'Argentine ?", + "Comment préparer une pâte à crêpes ?", + "Qui a gagné Roland-Garros en 2015 ?", + "Quel temps fera-t-il à Nice demain ?", + "Comment entretenir un aquarium d'eau douce ?", + "Quelle est la distance entre la Terre et la Lune ?", + "Quels sont les meilleurs vins de Bourgogne ?", + "Comment apprendre à jouer de la guitare ?", + "flimbo grastic wuvvle", + "trepax zondor kliffum", + "asdkj qwpoe zmxn", + "lorem quantum frabble", + "Combien de temps vit un chat domestique ?", + "What is the tallest mountain in Africa?", +] + +/** The embedding function, `embedTexts` in production, so callers pass it without a wrapper. */ +export type ProbeEmbedder = ( + texts: string[], + config: Config, + inputType: "query", + signal?: AbortSignal, +) => Promise + +const MAX_MEASURED_FLOORS = 4 +// Noise floors measured at search time for indexes built before ingestion stored one. +const measuredFloors = new Map>() + +/** + * Measure the noise floor of an index: embed the probes as queries and take the smallest exact + * distance to their nearest passage. Returns null for local-hash indexes and empty tables. + */ +export async function measureAbstentionCalibration( + table: Table, + config: Config, + embed: ProbeEmbedder, + signal?: AbortSignal, +): Promise { + if (config.embeddingProvider !== "transformers") { + return null + } + const vectors = await embed([...ABSTENTION_PROBES], config, "query", signal) + const distances: number[] = [] + for (const vector of vectors) { + const [nearest] = await table + .vectorSearch(vector) + .bypassVectorIndex() + .select(["id"]) + .limit(1) + .toArray() + if (typeof nearest?._distance === "number" && Number.isFinite(nearest._distance)) { + distances.push(nearest._distance) + } + } + if (distances.length === 0) { + return null + } + return { + schemaVersion: 1, + embeddingModel: config.embeddingModel, + probeCount: distances.length, + noiseFloorDistance: Math.min(...distances), + } +} + +/** + * The noise floor search applies: stored by ingestion, or measured once per process and index + * generation for an index built before ingestion stored it. A failed measurement keeps the + * policy's own distances. + */ +export function indexNoiseFloor( + table: Table, + config: Config, + manifest: IndexManifest | null, + generationKey: string, + embed: ProbeEmbedder, +): Promise { + if (config.embeddingProvider !== "transformers" || !manifest?.chunkCount) { + return Promise.resolve(null) + } + const stored = manifestNoiseFloor(manifest, config) + if (stored !== null) { + return Promise.resolve(stored) + } + const cacheKey = `${generationKey}\0${config.embeddingModel}` + let measured = measuredFloors.get(cacheKey) + if (!measured) { + measured = measureAbstentionCalibration(table, config, embed) + .then((calibration) => calibration?.noiseFloorDistance ?? null) + .catch(() => null) + measuredFloors.set(cacheKey, measured) + for (const key of measuredFloors.keys()) { + if (measuredFloors.size <= MAX_MEASURED_FLOORS) break + measuredFloors.delete(key) + } + } + return measured +} + +/** + * Advice for a semantic index built before ingestion stored its noise floor: every process then + * measures it on its first search, which cost 0.1 to 0.3 s on real bases. + */ +export function noiseFloorAdvice( + manifest: IndexManifest | null, + config: Config, + run: (args: string[]) => string, +): string | null { + if ( + config.embeddingProvider !== "transformers" || + !manifest?.chunkCount || + manifestNoiseFloor(manifest, config) !== null + ) { + return null + } + return `The index has no stored noise floor, so each process measures it on its first search. Run \`${run(["ingest"])}\` to store it.` +} + +/** The calibration stored with the index, when it was measured with the configured model. */ +export function manifestNoiseFloor(manifest: IndexManifest | null, config: Config): number | null { + const calibration = manifest?.abstentionCalibration + if ( + config.embeddingProvider !== "transformers" || + calibration?.schemaVersion !== 1 || + calibration.embeddingModel !== config.embeddingModel + ) { + return null + } + return calibration.noiseFloorDistance +} diff --git a/packages/ragmir-core/src/config-migration.test.ts b/packages/ragmir-core/src/config-migration.test.ts index 37e4296..6610172 100644 --- a/packages/ragmir-core/src/config-migration.test.ts +++ b/packages/ragmir-core/src/config-migration.test.ts @@ -77,6 +77,18 @@ describe("source-preserving migration", { timeout: integrationTimeout(20_000) }, }) }) + it("should report retired fields with a typed migration code before an upgrade", async () => { + const { root, configPath } = await fixture() + await writeFile(configPath, JSON.stringify({ privacyProfile: "private", accessLog: true })) + + await expect(loadConfig(root)).rejects.toMatchObject({ + name: "RagmirError", + code: "CONFIG_MIGRATION_REQUIRED", + retryable: false, + message: expect.stringContaining("Run `rgr upgrade`"), + }) + }) + it("should keep formerly blocked external operations disabled when migrating strict mode", async () => { const { root, configPath } = await fixture() await writeFile( diff --git a/packages/ragmir-core/src/config.ts b/packages/ragmir-core/src/config.ts index 92e9e74..a2eb05a 100644 --- a/packages/ragmir-core/src/config.ts +++ b/packages/ragmir-core/src/config.ts @@ -32,6 +32,7 @@ import { MAX_WORKLOAD_QUEUE_TIMEOUT_MS, SHARED_CONFIG_FILENAME, } from "./defaults.js" +import { RagmirError } from "./errors.js" import { isRecord } from "./guards.js" import type { Config, WorkloadLimit } from "./types.js" @@ -198,7 +199,7 @@ const rawConfigSchema = z type RawConfig = z.infer -export const RETIRED_CONFIG_KEYS = [ +const RETIRED_CONFIG_KEYS = [ "privacyProfile", "acceptedRisks", "redaction", @@ -304,7 +305,10 @@ export async function loadConfig(start = process.cwd()): Promise { const raw = await readMergedProjectConfig(projectConfig) const retired = retiredConfigKeys(raw) if (retired.length > 0) { - throw new Error( + // A typed code lets agents and MCP clients route to `rgr upgrade` instead of reporting an + // internal failure. + throw new RagmirError( + "CONFIG_MIGRATION_REQUIRED", `Retired configuration fields: ${retired.join(", ")}. Run \`rgr upgrade\` to back up and migrate the config and rebuild the index. Ragmir now preserves source text without masking; your consuming model controls where retrieved content is sent.`, ) } @@ -439,7 +443,7 @@ function configWithModelIdentityDefaults( } /** Budgets each retrieval profile chooses for fields the config leaves unset. */ -export const RETRIEVAL_PROFILE_BUDGETS: Readonly< +const RETRIEVAL_PROFILE_BUDGETS: Readonly< Record< Exclude, { topK: number; hybridTextScanLimit: number } diff --git a/packages/ragmir-core/src/doctor.ts b/packages/ragmir-core/src/doctor.ts index a4c2bdd..6a1b67d 100644 --- a/packages/ragmir-core/src/doctor.ts +++ b/packages/ragmir-core/src/doctor.ts @@ -1,5 +1,6 @@ import { existsSync } from "node:fs" import path from "node:path" +import { noiseFloorAdvice } from "./abstention-calibration.js" import { findProjectConfig, loadConfig, @@ -25,8 +26,9 @@ import { MCP_CONFIG_FILENAME, SKILL_NAMES, } from "./skill.js" +import { fragmentationWarning } from "./storage-maintenance.js" import { readIndexManifest, readIndexManifestHeader } from "./store.js" -import type { Config, DoctorOptions, DoctorReport } from "./types.js" +import type { Config, DoctorOptions, DoctorReport, IndexMaintenanceSnapshot } from "./types.js" export async function doctor( cwd = process.cwd(), @@ -129,6 +131,8 @@ export async function doctorWithConfig( config.retrievalProfile, ) : [] + const runCommand = (args: string[]): string => + command.display + (args.length > 0 ? ` ${args.join(" ")}` : "") const nextSteps = nextActions({ initialized, profileOverrides: profileOverrides.map( @@ -159,7 +163,9 @@ export async function doctorWithConfig( freshnessWarning, lexicalIndexCurrent, lexicalScanWarning, - run: (args) => command.display + (args.length > 0 ? ` ${args.join(" ")}` : ""), + fragments: manifest?.maintenance?.fragments, + noiseFloorAdvice: noiseFloorAdvice(manifest, config, runCommand), + run: runCommand, }) throwIfAborted(signal) @@ -235,6 +241,9 @@ interface NextActionInput { freshnessWarning: string | null lexicalIndexCurrent: boolean lexicalScanWarning: string | null + /** Fragment counts from the manifest's last maintenance snapshot. */ + fragments: IndexMaintenanceSnapshot["fragments"] | undefined + noiseFloorAdvice: string | null run: (args: string[]) => string } @@ -308,6 +317,15 @@ function nextActions(input: NextActionInput): string[] { steps.push(input.lexicalScanWarning) } + const storageWarning = fragmentationWarning(input.fragments, input.run) + if (storageWarning) { + steps.push(storageWarning) + } + + if (input.noiseFloorAdvice) { + steps.push(input.noiseFloorAdvice) + } + if (input.concentrationWarning) { steps.push(input.concentrationWarning) } diff --git a/packages/ragmir-core/src/errors.ts b/packages/ragmir-core/src/errors.ts index 5982fa1..bcb456e 100644 --- a/packages/ragmir-core/src/errors.ts +++ b/packages/ragmir-core/src/errors.ts @@ -1,6 +1,7 @@ export type RagmirErrorCode = | "ABORTED" | "CLIENT_CLOSED" + | "CONFIG_MIGRATION_REQUIRED" | "EVIDENCE_CHANGED" | "INDEX_BUSY" | "INDEX_UNAVAILABLE" diff --git a/packages/ragmir-core/src/ingest.ts b/packages/ragmir-core/src/ingest.ts index 9cdcec5..1c8eb84 100644 --- a/packages/ragmir-core/src/ingest.ts +++ b/packages/ragmir-core/src/ingest.ts @@ -1,4 +1,5 @@ import type { Connection } from "@lancedb/lancedb" +import { measureAbstentionCalibration } from "./abstention-calibration.js" import { summarizeChunkStats } from "./chunk-stats.js" import { chunkDocument, chunkLexicalText, chunkSearchText } from "./chunking.js" import { loadConfig } from "./config.js" @@ -66,6 +67,7 @@ import { writeIndexManifest, } from "./store.js" import type { + AbstentionCalibration, AuditOptions, AuditReport, Config, @@ -415,6 +417,13 @@ async function ingestUnlocked( } storageWarning = combineWarnings(storageWarning, maintenance.warning) await validateIngestionTable(state, manifest, config, connection) + const abstentionCalibration = + manifest.chunkCount > 0 + ? await measureIndexCalibration(tableName, config, connection, signal) + : null + if (abstentionCalibration) { + manifest = { ...manifest, abstentionCalibration } + } await writeEmptyTextFiles(emptyTextRecords(state), config) await writeIndexManifest(manifest, config, indexedFilesFromState(state)) try { @@ -478,6 +487,33 @@ async function ingestUnlocked( } } +/** + * Store where off-topic questions land in the new index, so search reads its noise floor instead of + * measuring it. Calibration is an optimization: a failure keeps the index and the policy distances. + */ +async function measureIndexCalibration( + tableName: string, + config: Config, + connection: Connection | undefined, + signal: AbortSignal | undefined, +): Promise { + if (config.embeddingProvider !== "transformers") { + return null + } + const table = await openRowsTableByName(tableName, config, connection) + if (!table) { + return null + } + try { + return await measureAbstentionCalibration(table, config, embedTexts, signal) + } catch { + throwIfAborted(signal) + return null + } finally { + closeRowsTable(table, config) + } +} + function combineWarnings(...warnings: Array): string | null { const present = warnings.filter((warning): warning is string => warning !== null) return present.length > 0 ? present.join(" ") : null @@ -955,6 +991,7 @@ export interface IngestionMetadataValidationOptions { fileRows: AsyncIterable } +/** @public Benchmarks import it from the build. */ export async function validateIngestionMetadata( options: IngestionMetadataValidationOptions, ): Promise { diff --git a/packages/ragmir-core/src/query.ts b/packages/ragmir-core/src/query.ts index 884361c..5de0e52 100644 --- a/packages/ragmir-core/src/query.ts +++ b/packages/ragmir-core/src/query.ts @@ -11,10 +11,11 @@ import { Operator, PhraseQuery, } from "@lancedb/lancedb" +import { indexNoiseFloor } from "./abstention-calibration.js" import { citationForCoordinates, stripCitationCoordinates } from "./citation.js" import { loadConfig } from "./config.js" import { MAX_SEARCH_TOP_K, VECTOR_DISTANCE_METRIC } from "./defaults.js" -import { embedText } from "./embeddings.js" +import { embedText, embedTexts } from "./embeddings.js" import { RagmirError } from "./errors.js" import { acquireGenerationReadLease } from "./generation-retention.js" import { indexFreshnessWarning } from "./index-diagnostics.js" @@ -33,6 +34,7 @@ import { abstentionFilter, fusionCandidateDepth, fuzzyMatchPrefilter, + isWeakAnswer, queryEvidence, queryTermCoverage, rankHybridRows, @@ -254,6 +256,13 @@ async function searchWithinGeneration( snapshot.manifest && lexicalPolicyCurrent(snapshot.manifest) ? queryTermFrequencies(table, queryTokens, snapshot).catch(() => null) : Promise.resolve(null) + const noiseFloorPromise = indexNoiseFloor( + table, + config, + snapshot.manifest, + `${snapshot.tableName}\0${snapshot.manifestFingerprint ?? ""}`, + embedTexts, + ) const lexicalCandidatesPromise = lexicalCandidateRows(table, sanitized.query, { ftsLimit: lexicalCandidateLimit(candidateDemand, config.retrievalProfile), fallbackBatchSize: config.hybridTextScanLimit, @@ -327,8 +336,10 @@ async function searchWithinGeneration( ) const isExactPathMatch = (row: SearchRow): boolean => lexicalCandidates.exactPathMatches.has(rowKey(row)) - const passesAbstention = abstentionFilter(evidence, rankingPolicy, termStatistics) - const hasWeakEvidence = weakEvidenceFilter(evidence, rankingPolicy, termStatistics) + const noiseFloor = await noiseFloorPromise + throwIfAborted(signal) + const passesAbstention = abstentionFilter(evidence, rankingPolicy, termStatistics, noiseFloor) + const hasWeakEvidence = weakEvidenceFilter(evidence, rankingPolicy, termStatistics, noiseFloor) const coverageOf = options.explain && termStatistics ? queryTermCoverage(evidence, termStatistics) : null const orderedRows = rankedRows.sort( @@ -343,7 +354,15 @@ async function searchWithinGeneration( : {}), isRelevant: (row) => isExactPathMatch(row) || passesAbstention(row), }) - const rows = diversity.rows + // A semantic answer whose leading passages are all weak holds no evidence: answer nothing rather + // than noise. An exact path match is never weak. + const rows = isWeakAnswer( + diversity.rows, + rankingPolicy, + (row) => !isExactPathMatch(row.row) && hasWeakEvidence(row.row), + ) + ? [] + : diversity.rows const [contextByRow, sourceStatuses] = await Promise.all([ contextChunksByRow(table, rows, contextRadius, retrievalPredicate), sourceStatusesForRows(config.projectRoot, rows), @@ -399,6 +418,7 @@ async function searchWithinGeneration( lexicalIndexedRows: lexicalCandidates.indexedRows, lexicalUnindexedRows: lexicalCandidates.unindexedRows, lexicalCoverage: lexicalCandidates.coverage, + noiseFloorDistance: noiseFloor, lexicalScanBatches: lexicalCandidates.scanBatches, diversityStrategy: "document-cap" as const, maxChunksPerDocument, @@ -508,9 +528,11 @@ async function queryTermFrequencies( measured .filter((token) => !generationCache.has(token)) .map(async (token) => { + // Selecting `_score` keeps LanceDB from projecting it implicitly, which logs a deprecation + // warning on stderr for every measured query word. const rows = await table .search(new MatchQuery(token, "searchText"), "fts", "searchText") - .select(["id"]) + .select(["id", "_score"]) .limit(frequencyLimit) .toArray() generationCache.set(token, rows.length) diff --git a/packages/ragmir-core/src/ranking.test.ts b/packages/ragmir-core/src/ranking.test.ts index d23e6a6..a35c312 100644 --- a/packages/ragmir-core/src/ranking.test.ts +++ b/packages/ragmir-core/src/ranking.test.ts @@ -2,8 +2,10 @@ import { describe, expect, it } from "vitest" import { abstentionFilter, candidatePassesAbstention, + effectiveVectorDistances, fusionCandidateDepth, fuzzyMatchPrefilter, + isWeakAnswer, type QueryTermStatistics, queryEvidence, queryTermCoverage, @@ -11,6 +13,7 @@ import { rankingPolicyFor, tokensAreFuzzyRelated, tokensAreLexicallyRelated, + weakEvidenceFilter, } from "./ranking.js" import { identifierParts, normalizeForMatch, tokenize } from "./text.js" @@ -27,7 +30,7 @@ describe("hybrid ranking", () => { const alpha = row("alpha.md", "Shared policy evidence.", { distance: 0.5, score: 3 }) const beta = row("beta.md", "Shared policy evidence.", { distance: 0.5, score: 3 }) const policy = rankingPolicyFor("local-hash", "balanced", 1) - expect(policy.version).toBe(7) + expect(policy.version).toBe(8) const forward = rankHybridRows("policy evidence", [beta, alpha], [alpha, beta], policy) const reversed = rankHybridRows("policy evidence", [alpha, beta], [beta, alpha], policy) @@ -295,6 +298,78 @@ function row( } } +describe("noise floor abstention", () => { + const e5 = rankingPolicyFor("transformers", "quality", 2, "intfloat/multilingual-e5-small") + const uncalibrated = rankingPolicyFor( + "transformers", + "quality", + 2, + "intfloat/multilingual-e5-base", + ) + const terms: QueryTermStatistics = { + documentCount: 100, + documentFrequencies: new Map([ + ["weather", 1], + ["forecast", 0], + ["tomorrow", 3], + ]), + } + + it("should cap strong and vocabulary-free distances with the index noise floor", () => { + expect(effectiveVectorDistances(e5, 0.27)).toEqual({ + maximumVectorDistance: 0.4, + strongVectorDistance: 0.27, + vocabularyFreeDistance: 0.24, + }) + expect(effectiveVectorDistances(uncalibrated, 0.31)).toEqual({ + maximumVectorDistance: 1.1, + strongVectorDistance: 0.31, + vocabularyFreeDistance: 0.31, + }) + expect(effectiveVectorDistances(e5, null)).toMatchObject({ maximumVectorDistance: 0.4 }) + const localHash = rankingPolicyFor("local-hash", "balanced", 1) + expect(effectiveVectorDistances(localHash, 0.2)).toMatchObject({ maximumVectorDistance: null }) + }) + + it("should require a passage nearer than any off-topic question when no word is known", () => { + const nonsense = queryEvidence("flimbo grastic wuvvle") + const unknown: QueryTermStatistics = { documentCount: 100, documentFrequencies: new Map() } + const near = row("near.md", "Release notes.", { distance: 0.28 }) + const noise = row("noise.md", "Release notes.", { distance: 0.6 }) + const passes = abstentionFilter(nonsense, uncalibrated, unknown, 0.31) + + expect(passes(near)).toBe(true) + expect(passes(noise)).toBe(false) + expect(abstentionFilter(nonsense, uncalibrated, unknown)(noise)).toBe(true) + }) + + it("should keep the policy admission distance for questions with known words", () => { + const evidence = queryEvidence("weather forecast tomorrow") + const semantic = row("semantic.md", "Unrelated release notes.", { distance: 0.33 }) + + expect(abstentionFilter(evidence, e5, terms, 0.3)(semantic)).toBe(true) + }) + + it("should treat a passage nearer than the floor as strong evidence", () => { + const evidence = queryEvidence("weather forecast tomorrow") + const weak = weakEvidenceFilter(evidence, uncalibrated, terms, 0.32) + + expect(weak(row("close.md", "Daily operations.", { distance: 0.2 }))).toBe(false) + expect(weak(row("far.md", "Daily operations.", { distance: 0.4 }))).toBe(true) + }) + + it("should refuse a semantic answer only when every leading result is weak", () => { + const flags = (values: boolean[]) => values.map((weak, index) => ({ index, weak })) + const isWeak = (entry: { weak: boolean }) => entry.weak + + expect(isWeakAnswer(flags([true, true, true, true, true, false]), e5, isWeak)).toBe(true) + expect(isWeakAnswer(flags([true, false, true]), e5, isWeak)).toBe(false) + expect(isWeakAnswer([], e5, isWeak)).toBe(false) + const localHash = rankingPolicyFor("local-hash", "balanced", 1) + expect(isWeakAnswer(flags([true, true]), localHash, isWeak)).toBe(false) + }) +}) + function rankedKeys(rows: Array<{ row: FixtureRow }>): string[] { return rows.map(({ row: value }) => `${value.relativePath}\0#${value.chunkIndex}`) } diff --git a/packages/ragmir-core/src/ranking.ts b/packages/ragmir-core/src/ranking.ts index 3208004..44fa56a 100644 --- a/packages/ragmir-core/src/ranking.ts +++ b/packages/ragmir-core/src/ranking.ts @@ -29,7 +29,7 @@ export interface RankedRow { } export interface RankingPolicy { - version: 7 + version: 8 embeddingProvider: EmbeddingProvider /** Semantic model whose calibrated evidence profile applies; null for local-hash. */ embeddingModel: string | null @@ -90,7 +90,8 @@ const TRANSFORMERS_MAXIMUM_VECTOR_DISTANCE = 1.1 // Calibrated on French and English golden sets plus out-of-scope and nonsense questions. e5 packs // every text into a narrow distance band (relevant passages up to 0.36, unrelated ones from 0.26), // so its vector channel stays permissive and weak evidence relies more on query-word coverage. -// mxbai separates them (relevant passages mostly under 1.1, unrelated ones above 1.0). +// mxbai separates them (relevant passages mostly under 1.1, unrelated ones above 1.0). These +// absolute distances are upper bounds: each index also caps them with its own noise floor. const MODEL_EVIDENCE_PROFILES: Readonly> = { "intfloat/multilingual-e5-small": { maximumVectorDistance: 0.4, @@ -99,15 +100,18 @@ const MODEL_EVIDENCE_PROFILES: Readonly> = { minimumLexicalCoverage: 0.3, weakEvidenceCoverage: 0.3, }, + // A coverage of 0.5 flagged a third of the correct passages of a TypeScript repository as weak: + // natural-language questions share few words with code. 0.3 flagged 6 of 96. "mixedbread-ai/mxbai-embed-xsmall-v1": { maximumVectorDistance: TRANSFORMERS_MAXIMUM_VECTOR_DISTANCE, strongVectorDistance: 0.9, vocabularyFreeDistance: 0.85, minimumLexicalCoverage: 0.3, - weakEvidenceCoverage: 0.5, + weakEvidenceCoverage: 0.3, }, } // Uncalibrated models keep the historical rule: any related query word, or a distance up to 1.1. +// The index's noise floor then decides which of their passages count as strong evidence. const DEFAULT_TRANSFORMERS_PROFILE: EvidenceProfile = { maximumVectorDistance: TRANSFORMERS_MAXIMUM_VECTOR_DISTANCE, strongVectorDistance: null, @@ -149,7 +153,7 @@ export function rankingPolicyFor( DEFAULT_TRANSFORMERS_PROFILE) : LOCAL_HASH_PROFILE return { - version: 7, + version: 8, embeddingProvider, embeddingModel: semantic ? (embeddingModel ?? null) : null, retrievalProfile, @@ -287,20 +291,64 @@ export function candidatePassesAbstention( row: RankingRow, policy: RankingPolicy, terms: QueryTermStatistics | null = null, + noiseFloor: number | null = null, ): boolean { const evidence = typeof evidenceOrQuery === "string" ? queryEvidence(evidenceOrQuery) : evidenceOrQuery - return abstentionFilter(evidence, policy, terms)(row) + return abstentionFilter(evidence, policy, terms, noiseFloor)(row) +} + +/** + * The policy's vector distances with the index's noise floor applied: the distance at which the + * nearest passage of an off-topic probe question sits. A model packs every text into its own + * band, and a corpus shifts it, so only a passage nearer than any off-topic question counts as + * strong evidence, and a question with no known word needs one. The admission distance stays the + * policy's: capping it removed correct passages from real developer questions. Local-hash vectors + * carry no meaning and keep the policy. + */ +export function effectiveVectorDistances( + policy: RankingPolicy, + noiseFloor: number | null, +): Pick< + RankingPolicy, + "maximumVectorDistance" | "strongVectorDistance" | "vocabularyFreeDistance" +> { + if (noiseFloor === null || policy.embeddingProvider === "local-hash") { + return policy + } + const capped = (distance: number | null): number => + distance === null ? noiseFloor : Math.min(distance, noiseFloor) + return { + maximumVectorDistance: policy.maximumVectorDistance, + strongVectorDistance: capped(policy.strongVectorDistance), + vocabularyFreeDistance: capped(policy.vocabularyFreeDistance ?? policy.maximumVectorDistance), + } +} + +/** How many leading results must all be weak before semantic search answers nothing. */ +const WEAK_ANSWER_DEPTH = 5 + +/** + * Whether a semantic answer holds no real evidence: every leading result is weak. Off-topic and + * near-domain questions without an answer typically land here. Local-hash search keeps such + * results, since its weak flag was never calibrated to refuse. + */ +export function isWeakAnswer( + rows: readonly Row[], + policy: RankingPolicy, + isWeak: (row: Row) => boolean, +): boolean { + if (policy.embeddingProvider === "local-hash" || rows.length === 0) { + return false + } + return rows.slice(0, WEAK_ANSWER_DEPTH).every(isWeak) } /** * Whether no query word occurs in the corpus, exactly or through a fuzzy keyword match. Only * vector evidence can then support a passage, and nonsense or unrelated questions get none. */ -export function vocabularyFree( - evidence: QueryEvidence, - terms: QueryTermStatistics | null, -): boolean { +function vocabularyFree(evidence: QueryEvidence, terms: QueryTermStatistics | null): boolean { return ( terms !== null && evidence.anchors.length === 0 && @@ -324,6 +372,7 @@ export function abstentionFilter( evidence: QueryEvidence, policy: RankingPolicy, terms: QueryTermStatistics | null = null, + noiseFloor: number | null = null, ): (row: RankingRow) => boolean { const hasLexicalEvidence = lexicalEvidenceMatcher(evidence) if (evidence.anchors.length > 0) { @@ -333,9 +382,10 @@ export function abstentionFilter( if (policy.embeddingProvider === "local-hash") { return noKnownWords ? () => false : (row) => hasLexicalEvidence(row.searchText) } - const maximumVectorDistance = policy.maximumVectorDistance + const distances = effectiveVectorDistances(policy, noiseFloor) + const maximumVectorDistance = distances.maximumVectorDistance if (noKnownWords) { - const required = policy.vocabularyFreeDistance ?? maximumVectorDistance + const required = distances.vocabularyFreeDistance ?? maximumVectorDistance return (row) => required !== null && rowDistance(row) <= required } const coverage = @@ -355,12 +405,13 @@ export function weakEvidenceFilter( evidence: QueryEvidence, policy: RankingPolicy, terms: QueryTermStatistics | null, + noiseFloor: number | null = null, ): (row: RankingRow) => boolean { if (terms === null || evidence.anchors.length > 0 || evidence.tokens.length === 0) { return () => false } const coverage = queryTermCoverage(evidence, terms) - const strong = policy.strongVectorDistance + const strong = effectiveVectorDistances(policy, noiseFloor).strongVectorDistance return (row) => coverage(row.searchText) < policy.weakEvidenceCoverage && !(strong !== null && rowDistance(row) <= strong) diff --git a/packages/ragmir-core/src/storage-maintenance.test.ts b/packages/ragmir-core/src/storage-maintenance.test.ts new file mode 100644 index 0000000..aaeb840 --- /dev/null +++ b/packages/ragmir-core/src/storage-maintenance.test.ts @@ -0,0 +1,18 @@ +import { describe, expect, it } from "vitest" +import { fragmentationWarning } from "./storage-maintenance.js" + +const run = (args: string[]): string => `rgr ${args.join(" ")}` + +describe("fragmentationWarning", () => { + it("should advise compaction for an index left in many small fragments", () => { + expect(fragmentationWarning({ total: 709, small: 709, smallRatio: 1 }, run)).toBe( + "The index is split into 709 table fragments, which slows every search. Run `rgr storage optimize` to compact it.", + ) + }) + + it("should stay silent for a compact index, mostly large fragments, or no snapshot", () => { + expect(fragmentationWarning({ total: 9, small: 9, smallRatio: 1 }, run)).toBeNull() + expect(fragmentationWarning({ total: 120, small: 12, smallRatio: 0.1 }, run)).toBeNull() + expect(fragmentationWarning(undefined, run)).toBeNull() + }) +}) diff --git a/packages/ragmir-core/src/storage-maintenance.ts b/packages/ragmir-core/src/storage-maintenance.ts index fb4592f..637e3d3 100644 --- a/packages/ragmir-core/src/storage-maintenance.ts +++ b/packages/ragmir-core/src/storage-maintenance.ts @@ -396,6 +396,24 @@ function maintenanceReasons( return reasons } +/** + * Advice for an index left in many small fragments, for example by a release that compacted only + * past 100,000 rows. Such a 10,000-chunk index searched five times slower until it was compacted. + */ +export function fragmentationWarning( + fragments: StorageMaintenanceReport["fragments"] | undefined, + run: (args: string[]) => string, +): string | null { + if ( + fragments === undefined || + fragments.total < AUTOMATIC_COMPACTION_FRAGMENT_COUNT || + fragments.smallRatio < SMALL_FRAGMENT_RATIO_THRESHOLD + ) { + return null + } + return `The index is split into ${fragments.total} table fragments, which slows every search. Run \`${run(["storage", "optimize"])}\` to compact it.` +} + function fragmentationRequiresCompaction( health: TableHealth, automaticCompactionEligible: boolean, diff --git a/packages/ragmir-core/src/types.ts b/packages/ragmir-core/src/types.ts index 0d47e3a..fc26ee5 100644 --- a/packages/ragmir-core/src/types.ts +++ b/packages/ragmir-core/src/types.ts @@ -190,6 +190,7 @@ export interface IndexManifest { staleFiles?: IndexManifestStaleFile[] health?: IndexHealthSnapshot maintenance?: IndexMaintenanceSnapshot + abstentionCalibration?: AbstentionCalibration qualityReport?: IndexQualityReport } @@ -244,6 +245,18 @@ export interface IndexFreshnessReport { error: string | null } +/** + * Where off-topic questions land in this index: the smallest distance between a fixed set of + * off-topic probe questions and their nearest passage. Search only accepts vector evidence nearer + * than this noise floor. + */ +export interface AbstentionCalibration { + schemaVersion: 1 + embeddingModel: string + probeCount: number + noiseFloorDistance: number +} + export interface IndexMaintenanceSnapshot { schemaVersion: 1 checkedAt: string @@ -747,6 +760,8 @@ export interface SearchScoreExplanation { matchedTerms: string[] /** IDF-weighted share of query words in the passage; null without word statistics. */ queryTermCoverage: number | null + /** Distance of the nearest passage to off-topic probe questions; null for local-hash. */ + noiseFloorDistance: number | null } export interface ExpandCitationOptions extends OperationOptions { diff --git a/tests/public-api-consumer/consumer.ts b/tests/public-api-consumer/consumer.ts index 47996f0..4a387d1 100644 --- a/tests/public-api-consumer/consumer.ts +++ b/tests/public-api-consumer/consumer.ts @@ -97,6 +97,7 @@ const errorCode: RagmirErrorCode = "TIMEOUT" const indexErrorCode: RagmirErrorCode = "INDEX_UNAVAILABLE" const overloadedErrorCode: RagmirErrorCode = "OVERLOADED" const changedEvidenceErrorCode: RagmirErrorCode = "EVIDENCE_CHANGED" +const migrationErrorCode: RagmirErrorCode = "CONFIG_MIGRATION_REQUIRED" void semanticResult void disposedModels @@ -106,5 +107,6 @@ void errorCode void indexErrorCode void overloadedErrorCode void changedEvidenceErrorCode +void migrationErrorCode void createMcpServer(cwd) From ebfcdc1a06554ac842ba262c4f9ada9ea7f1aecc Mon Sep 17 00:00:00 2001 From: Jean-Baptiste THERY Date: Sun, 1 Nov 2026 19:17:54 +0700 Subject: [PATCH 2/5] refactor: remove dead code and duplicates found by Knip - share one nearest-rank percentile in Core and one in the benchmark library instead of eight copies - read JSON Lines files through one helper for the ingestion snapshot and the fingerprint cache, and document why the append-only journal tolerates a torn last line - validate the ingestion run state through the run metadata check instead of a copy of it - delete the unused staged-manifest writer and fallback gitignore entry, unexport internal constants and helpers, and mark the deprecated command aliases - render the 404 and 500 pages from one error-page component that reuses the logo component --- .../ragmir-core/benchmarks/cli-startup.mjs | 5 +- .../benchmarks/concurrency-worker.mjs | 8 +- .../ragmir-core/benchmarks/lib/metrics.mjs | 22 +++--- .../benchmarks/local-hash-experiment.mjs | 6 +- .../benchmarks/reranker-experiment.mjs | 6 +- packages/ragmir-core/benchmarks/run.mjs | 14 +--- packages/ragmir-core/benchmarks/status.mjs | 6 +- packages/ragmir-core/src/chunk-stats.ts | 6 +- packages/ragmir-core/src/defaults.ts | 10 +-- packages/ragmir-core/src/evaluate.ts | 11 +-- packages/ragmir-core/src/files.ts | 12 +-- packages/ragmir-core/src/git-hooks.ts | 7 +- packages/ragmir-core/src/gitignore.ts | 2 +- packages/ragmir-core/src/index-write-lock.ts | 2 +- packages/ragmir-core/src/ingestion-state.ts | 77 +++---------------- packages/ragmir-core/src/json-lines.ts | 37 +++++++++ packages/ragmir-core/src/ocr-cache.ts | 2 +- packages/ragmir-core/src/ocr.ts | 2 +- packages/ragmir-core/src/package-manager.ts | 2 + packages/ragmir-core/src/quality-report.ts | 2 +- packages/ragmir-core/src/query-sanitizer.ts | 2 +- packages/ragmir-core/src/skill.ts | 2 +- .../src/source-fingerprint-cache.ts | 34 ++------ packages/ragmir-core/src/source-weights.ts | 2 +- packages/ragmir-core/src/statistics.ts | 7 ++ .../ragmir-core/src/test-support/timeouts.ts | 2 +- packages/ragmir-core/src/vector-index.ts | 6 +- packages/ragmir-core/src/workload.ts | 2 +- .../src/components/error-page.astro | 71 +++++++++++++++++ .../ragmir-landing/src/components/ui/tabs.tsx | 2 +- packages/ragmir-landing/src/pages/404.astro | 72 +++-------------- packages/ragmir-landing/src/pages/500.astro | 72 +++-------------- 32 files changed, 207 insertions(+), 306 deletions(-) create mode 100644 packages/ragmir-core/src/json-lines.ts create mode 100644 packages/ragmir-core/src/statistics.ts create mode 100644 packages/ragmir-landing/src/components/error-page.astro diff --git a/packages/ragmir-core/benchmarks/cli-startup.mjs b/packages/ragmir-core/benchmarks/cli-startup.mjs index d00d628..369a0e1 100644 --- a/packages/ragmir-core/benchmarks/cli-startup.mjs +++ b/packages/ragmir-core/benchmarks/cli-startup.mjs @@ -2,6 +2,7 @@ import { spawnSync } from "node:child_process" import path from "node:path" import { performance } from "node:perf_hooks" import { fileURLToPath } from "node:url" +import { percentile } from "./lib/metrics.mjs" const benchmarkRoot = path.dirname(fileURLToPath(import.meta.url)) const packageRoot = path.resolve(benchmarkRoot, "..") @@ -88,10 +89,6 @@ function measureCli(arguments_) { } } -function percentile(sorted, quantile) { - return sorted[Math.min(sorted.length - 1, Math.ceil(sorted.length * quantile) - 1)] -} - function positiveInteger(value) { const parsed = Number.parseInt(value, 10) if (!Number.isInteger(parsed) || parsed <= 0) { diff --git a/packages/ragmir-core/benchmarks/concurrency-worker.mjs b/packages/ragmir-core/benchmarks/concurrency-worker.mjs index ba5e63b..caabc63 100644 --- a/packages/ragmir-core/benchmarks/concurrency-worker.mjs +++ b/packages/ragmir-core/benchmarks/concurrency-worker.mjs @@ -2,6 +2,7 @@ import { performance } from "node:perf_hooks" import { createRagmirClient } from "../dist/index.js" import { loadConfig } from "../dist/config.js" import { workloadSnapshot } from "../dist/workload.js" +import { percentile } from "./lib/metrics.mjs" const root = requiredEnvironment("RAGMIR_CONCURRENCY_ROOT") const requests = positiveInteger(process.env.RAGMIR_CONCURRENCY_REQUESTS ?? "100") @@ -79,13 +80,6 @@ function summarize(sorted) { } } -function percentile(sorted, ratio) { - if (sorted.length === 0) { - return 0 - } - return sorted[Math.min(sorted.length - 1, Math.ceil(sorted.length * ratio) - 1)] ?? 0 -} - function requiredEnvironment(name) { const value = process.env[name] if (!value) { diff --git a/packages/ragmir-core/benchmarks/lib/metrics.mjs b/packages/ragmir-core/benchmarks/lib/metrics.mjs index 2c6e77f..addcdc3 100644 --- a/packages/ragmir-core/benchmarks/lib/metrics.mjs +++ b/packages/ragmir-core/benchmarks/lib/metrics.mjs @@ -24,24 +24,24 @@ export function stableJson(value) { return JSON.stringify(value) } -export function percentile(values, quantile) { - if (values.length === 0) { - return 0 - } - const sorted = [...values].sort((left, right) => left - right) - const index = Math.max(0, Math.ceil(quantile * sorted.length) - 1) - return sorted[index] ?? 0 +/** + * Nearest-rank percentile of ascending values, the same rule as Core's `percentile`: the smallest + * value with at least `quantile` of the values at or below it. Empty input gives 0. + */ +export function percentile(sortedValues, quantile) { + return sortedValues[Math.max(0, Math.ceil(quantile * sortedValues.length) - 1)] ?? 0 } -export function summarizeSamples(values) { +function summarizeSamples(values) { const total = values.reduce((sum, value) => sum + value, 0) + const sorted = [...values].sort((left, right) => left - right) return { count: values.length, minMs: values.length === 0 ? 0 : Math.min(...values), meanMs: values.length === 0 ? 0 : total / values.length, - p50Ms: percentile(values, 0.5), - p95Ms: percentile(values, 0.95), - p99Ms: percentile(values, 0.99), + p50Ms: percentile(sorted, 0.5), + p95Ms: percentile(sorted, 0.95), + p99Ms: percentile(sorted, 0.99), maxMs: values.length === 0 ? 0 : Math.max(...values), } } diff --git a/packages/ragmir-core/benchmarks/local-hash-experiment.mjs b/packages/ragmir-core/benchmarks/local-hash-experiment.mjs index b7c3701..857b148 100644 --- a/packages/ragmir-core/benchmarks/local-hash-experiment.mjs +++ b/packages/ragmir-core/benchmarks/local-hash-experiment.mjs @@ -4,7 +4,7 @@ import path from "node:path" import { performance } from "node:perf_hooks" import { fileURLToPath } from "node:url" import { tokenize } from "../dist/text.js" -import { environmentMetadata } from "./lib/metrics.mjs" +import { environmentMetadata, percentile } from "./lib/metrics.mjs" const dimensions = 384 const tokenHashSeed = 0x811c9dc5 @@ -219,10 +219,6 @@ function delta(candidate, baseline) { return candidate - baseline } -function percentile(sorted, quantile) { - return sorted[Math.min(sorted.length - 1, Math.max(0, Math.ceil(sorted.length * quantile) - 1))] -} - function benchmarkTexts() { const samples = [ "Production approval requires an immutable audit trail and cited evidence.", diff --git a/packages/ragmir-core/benchmarks/reranker-experiment.mjs b/packages/ragmir-core/benchmarks/reranker-experiment.mjs index 5763f48..ae00e7a 100644 --- a/packages/ragmir-core/benchmarks/reranker-experiment.mjs +++ b/packages/ragmir-core/benchmarks/reranker-experiment.mjs @@ -11,7 +11,7 @@ import { } from "@huggingface/transformers" import { createRagmirClient } from "../dist/index.js" import { CORPUS_PRESETS, generateCorpus } from "./lib/corpus.mjs" -import { environmentMetadata } from "./lib/metrics.mjs" +import { environmentMetadata, percentile } from "./lib/metrics.mjs" const benchmarkPath = fileURLToPath(import.meta.url) const packageRoot = path.resolve(path.dirname(benchmarkPath), "..") @@ -360,10 +360,6 @@ function latencySummary(values) { } } -function percentile(sorted, quantile) { - return sorted[Math.max(0, Math.ceil(sorted.length * quantile) - 1)] ?? 0 -} - function mean(values) { return values.length === 0 ? 0 : values.reduce((sum, value) => sum + value, 0) / values.length } diff --git a/packages/ragmir-core/benchmarks/run.mjs b/packages/ragmir-core/benchmarks/run.mjs index 094f0a2..8ad9b96 100644 --- a/packages/ragmir-core/benchmarks/run.mjs +++ b/packages/ragmir-core/benchmarks/run.mjs @@ -20,6 +20,7 @@ import { environmentMetadata, measureOperation, measureSeries, + percentile, settleMeasurement, } from "./lib/metrics.mjs" @@ -507,9 +508,9 @@ async function measureConcurrency({ client, queries, concurrency, total }) { total, wallMs, throughputPerSecond: wallMs === 0 ? 0 : (total * 1_000) / wallMs, - p50Ms: percentileFromSorted(durations, 0.5), - p95Ms: percentileFromSorted(durations, 0.95), - p99Ms: percentileFromSorted(durations, 0.99), + p50Ms: percentile(durations, 0.5), + p95Ms: percentile(durations, 0.95), + p99Ms: percentile(durations, 0.99), } } @@ -631,13 +632,6 @@ function normalizeProvider(value) { return value } -function percentileFromSorted(values, quantile) { - if (values.length === 0) { - return 0 - } - return values[Math.max(0, Math.ceil(values.length * quantile) - 1)] ?? 0 -} - function parseArguments(values) { const parsed = {} for (let index = 0; index < values.length; index += 1) { diff --git a/packages/ragmir-core/benchmarks/status.mjs b/packages/ragmir-core/benchmarks/status.mjs index 2adc307..393f62b 100644 --- a/packages/ragmir-core/benchmarks/status.mjs +++ b/packages/ragmir-core/benchmarks/status.mjs @@ -6,7 +6,7 @@ import os from "node:os" import path from "node:path" import { fileURLToPath } from "node:url" import { performance } from "node:perf_hooks" -import { environmentMetadata } from "./lib/metrics.mjs" +import { environmentMetadata, percentile } from "./lib/metrics.mjs" const SAMPLE_COUNT = 100 const WARMUP_COUNT = 10 @@ -136,7 +136,3 @@ async function replaceChunkCount(projectRoot, chunkCount) { } await writeFile(manifestPath, `${JSON.stringify(manifest, null, 2)}\n`) } - -function percentile(sorted, quantile) { - return sorted[Math.max(0, Math.ceil(sorted.length * quantile) - 1)] ?? 0 -} diff --git a/packages/ragmir-core/src/chunk-stats.ts b/packages/ragmir-core/src/chunk-stats.ts index ea4b6bf..8304693 100644 --- a/packages/ragmir-core/src/chunk-stats.ts +++ b/packages/ragmir-core/src/chunk-stats.ts @@ -1,3 +1,4 @@ +import { percentile } from "./statistics.js" import type { ChunkStats } from "./types.js" interface ChunkShape { @@ -33,8 +34,3 @@ export function summarizeChunkStats(chunks: readonly ChunkShape[]): ChunkStats { contextualRatio: contextualChunks / chunks.length, } } - -function percentile(sortedValues: number[], quantile: number): number { - const index = Math.ceil(quantile * sortedValues.length) - 1 - return sortedValues[Math.max(0, index)] ?? 0 -} diff --git a/packages/ragmir-core/src/defaults.ts b/packages/ragmir-core/src/defaults.ts index 282d386..e56f200 100644 --- a/packages/ragmir-core/src/defaults.ts +++ b/packages/ragmir-core/src/defaults.ts @@ -3,7 +3,7 @@ import type { Config } from "./types.js" export const RAGMIR_DIR = ".ragmir" export const LEGACY_KB_DIR = ".kb" export const LEGACY_PRIVATE_DIR = "private" -export const RAGMIR_RAW_DIR = `${RAGMIR_DIR}/raw` +const RAGMIR_RAW_DIR = `${RAGMIR_DIR}/raw` export const CONFIG_PATH = `${RAGMIR_DIR}/config.json` /** Optional project-level config meant to be committed; `.ragmir/config.json` overrides its keys. */ export const SHARED_CONFIG_FILENAME = "ragmir.config.json" @@ -23,6 +23,7 @@ export const MAX_INGEST_CHUNK_WINDOW = 8_192 export const MAX_INGEST_CHUNKS_PER_FILE = 65_536 export const MAX_INGEST_VECTOR_BYTES_PER_FILE = 256 * 1_024 * 1_024 export const MAX_SEARCH_TOP_K = 100 +/** A document can fill at most one full result page. @alias */ export const MAX_CHUNKS_PER_DOCUMENT = MAX_SEARCH_TOP_K export const MAX_HYBRID_TEXT_SCAN_LIMIT = 10_000 export const MAX_CONFIG_ARRAY_ITEMS = 10_000 @@ -39,9 +40,9 @@ export const MAX_WORKLOAD_QUEUE_TIMEOUT_MS = 900_000 export const MAX_CONFIG_NAME_CHARACTERS = 120 export const MAX_SOURCE_WEIGHT = 10 export const MAX_SEARCH_SCOPES = 64 -export const DEFAULT_EMBEDDING_MODEL = "intfloat/multilingual-e5-small" -export const DEFAULT_EMBEDDING_MODEL_REVISION = "614241f622f53c4eeff9890bdc4f31cfecc418b3" -export const KNOWN_EMBEDDING_MODEL_REVISIONS: Readonly> = { +const DEFAULT_EMBEDDING_MODEL = "intfloat/multilingual-e5-small" +const DEFAULT_EMBEDDING_MODEL_REVISION = "614241f622f53c4eeff9890bdc4f31cfecc418b3" +const KNOWN_EMBEDDING_MODEL_REVISIONS: Readonly> = { [DEFAULT_EMBEDDING_MODEL]: DEFAULT_EMBEDDING_MODEL_REVISION, "mixedbread-ai/mxbai-embed-xsmall-v1": "e6ac24e5d6efb8782b59de1647b3ececb4ece94e", } @@ -59,7 +60,6 @@ export function defaultEmbeddingModelRevision(model: string): string { export const RAGMIR_GITIGNORE_ENTRY = `${RAGMIR_DIR}/` export const LEGACY_KB_GITIGNORE_ENTRY = `${LEGACY_KB_DIR}/` export const LEGACY_PRIVATE_GITIGNORE_ENTRY = `${LEGACY_PRIVATE_DIR}/` -export const LEGACY_PRIVATE_GITIGNORE_FALLBACK_ENTRY = `${LEGACY_PRIVATE_DIR}/**` export const DEFAULT_CONFIG: Omit = { retrievalProfile: "balanced", diff --git a/packages/ragmir-core/src/evaluate.ts b/packages/ragmir-core/src/evaluate.ts index c734860..8bfa58a 100644 --- a/packages/ragmir-core/src/evaluate.ts +++ b/packages/ragmir-core/src/evaluate.ts @@ -13,6 +13,7 @@ import { fingerprintIndexManifest, fingerprintQualityReport } from "./quality-re import { searchWithConfig } from "./query.js" import { rankingPolicyFingerprint, rankingPolicyFor } from "./ranking.js" import { canonicalUserPath } from "./source-paths.js" +import { percentile } from "./statistics.js" import { closeIndexReadSnapshot, closeStoreConnection, @@ -245,7 +246,7 @@ export async function evaluateGoldenQueries(options: EvaluationOptions): Promise return evaluateGoldenQueriesWithConfig(options, config) } -export async function evaluateGoldenQueriesWithConfig( +async function evaluateGoldenQueriesWithConfig( options: EvaluationOptions, config: Config, ): Promise { @@ -974,14 +975,6 @@ function citationForResult(result: { citation: string }): string { return result.citation } -function percentile(sortedValues: number[], quantile: number): number { - if (sortedValues.length === 0) { - return 0 - } - const index = Math.ceil(quantile * sortedValues.length) - 1 - return sortedValues[Math.max(0, index)] ?? 0 -} - function mean(values: number[]): number { return values.length === 0 ? 0 : values.reduce((sum, value) => sum + value, 0) / values.length } diff --git a/packages/ragmir-core/src/files.ts b/packages/ragmir-core/src/files.ts index 05ab719..6c7015f 100644 --- a/packages/ragmir-core/src/files.ts +++ b/packages/ragmir-core/src/files.ts @@ -109,7 +109,7 @@ const DEFAULT_SUPPORTED_FILE_NAMES = new Set([ "procfile", "rakefile", ]) -export const DEFAULT_FAST_GLOB_IGNORES = [ +const DEFAULT_FAST_GLOB_IGNORES = [ "**/.git/**", "**/node_modules/**", `**/${LEGACY_KB_DIR}/**`, @@ -119,7 +119,7 @@ export const DEFAULT_FAST_GLOB_IGNORES = [ * Tool state, dependency caches, lockfiles, and generated bundles that drown out project knowledge. * They are skipped unless `defaultExcludes` is false or a source entry names the directory itself. */ -export const DEFAULT_EXCLUDED_DIRECTORIES = [ +const DEFAULT_EXCLUDED_DIRECTORIES = [ ".agents", ".angular", ".cache", @@ -155,7 +155,7 @@ export const DEFAULT_EXCLUDED_DIRECTORIES = [ "DerivedData", "Pods", ] as const -export const DEFAULT_EXCLUDED_FILES = [ +const DEFAULT_EXCLUDED_FILES = [ "*.map", "*.min.css", "*.min.js", @@ -221,7 +221,7 @@ export interface SourceInventoryOptions extends OperationOptions { writeFingerprintCache?: boolean } -export const DEFAULT_SUPPORTED_EXTENSIONS = new Set([ +const DEFAULT_SUPPORTED_EXTENSIONS = new Set([ ".atom", ".adoc", ".astro", @@ -778,7 +778,7 @@ async function mapLimit( await Promise.all(workers) } -export function supportedExtensions(config: Config): Set { +function supportedExtensions(config: Config): Set { return new Set([ ...DEFAULT_SUPPORTED_EXTENSIONS, ...(config.imageOcrCommand.length > 0 ? OCR_IMAGE_EXTENSIONS : []), @@ -929,7 +929,7 @@ function sourcePattern(projectRoot: string, input: string): string { return canonicalSourcePath(pattern).replace(/^(?:\.\/)+/u, "") } -export function isSensitiveFilePath(absolutePath: string): boolean { +function isSensitiveFilePath(absolutePath: string): boolean { const baseName = path.basename(absolutePath).toLowerCase() const extension = path.extname(absolutePath).toLowerCase() return ( diff --git a/packages/ragmir-core/src/git-hooks.ts b/packages/ragmir-core/src/git-hooks.ts index 9248f6d..34f38f6 100644 --- a/packages/ragmir-core/src/git-hooks.ts +++ b/packages/ragmir-core/src/git-hooks.ts @@ -6,12 +6,7 @@ import { RAGMIR_DIR } from "./defaults.js" import { RGR_RUNNER_FILENAME } from "./package-manager.js" import { canonicalSourcePath } from "./source-paths.js" -export const GIT_HOOK_NAMES = [ - "post-checkout", - "post-commit", - "post-merge", - "post-rewrite", -] as const +const GIT_HOOK_NAMES = ["post-checkout", "post-commit", "post-merge", "post-rewrite"] as const export type GitHookName = (typeof GIT_HOOK_NAMES)[number] const BLOCK_START = "# >>> ragmir auto-ingest >>>" diff --git a/packages/ragmir-core/src/gitignore.ts b/packages/ragmir-core/src/gitignore.ts index 8d4df22..8b12c3f 100644 --- a/packages/ragmir-core/src/gitignore.ts +++ b/packages/ragmir-core/src/gitignore.ts @@ -3,7 +3,7 @@ import { readFile, writeFile } from "node:fs/promises" import path from "node:path" import { RAGMIR_GITIGNORE_ENTRY } from "./defaults.js" -export const RAGMIR_GITIGNORE_ENTRIES = [RAGMIR_GITIGNORE_ENTRY] +const RAGMIR_GITIGNORE_ENTRIES = [RAGMIR_GITIGNORE_ENTRY] export async function ensureRagmirGitignore( cwd = process.cwd(), diff --git a/packages/ragmir-core/src/index-write-lock.ts b/packages/ragmir-core/src/index-write-lock.ts index ca3b9c9..0ba9b5f 100644 --- a/packages/ragmir-core/src/index-write-lock.ts +++ b/packages/ragmir-core/src/index-write-lock.ts @@ -8,7 +8,7 @@ import { isRecord } from "./guards.js" import { throwIfAborted } from "./operation.js" import { ensurePrivateDirectory, hardenPrivateFile } from "./permissions.js" -export const INDEX_WRITE_LOCK_DIRECTORY = ".writer-lock" +const INDEX_WRITE_LOCK_DIRECTORY = ".writer-lock" const RECOVERY_LOCK_DIRECTORY = ".writer-lock-recovery" const OWNER_FILENAME = "owner.json" const DEFAULT_WAIT_TIMEOUT_MS = 30_000 diff --git a/packages/ragmir-core/src/ingestion-state.ts b/packages/ragmir-core/src/ingestion-state.ts index 6a5ab7e..6cd4cae 100644 --- a/packages/ragmir-core/src/ingestion-state.ts +++ b/packages/ragmir-core/src/ingestion-state.ts @@ -4,10 +4,10 @@ import { open as openFile, readdir, readFile, rm } from "node:fs/promises" import path from "node:path" import { syncDirectory, writePrivateFileAtomic } from "./durable-file.js" import { isRecord } from "./guards.js" +import { acceptJsonLines } from "./json-lines.js" import { ensurePrivateDirectory, hardenPrivateFile } from "./permissions.js" import type { Config, - IndexManifest, IndexManifestFile, IngestionFileStage, IngestionProgress, @@ -305,14 +305,6 @@ export async function compactIngestionState( runtime.dirtyFiles.clear() } -export async function writeStagedIndexManifest( - manifest: IndexManifest, - runId: string, - config: Config, -): Promise { - await writePrivateJsonAtomic(stagedManifestPath(runId, config), manifest, config.storageDir) -} - export async function removeStagedIndexManifest(runId: string, config: Config): Promise { await rm(stagedManifestPath(runId, config), { force: true }) } @@ -445,6 +437,8 @@ async function applyIngestionJournal( runtime: IngestionRuntime, snapshotUpdatedAt: string, ): Promise { + // Unlike the file snapshot, the append-only journal skips empty lines and ignores a final line + // without a newline: an append interrupted by a crash must not discard the entries before it. const stream = createReadStream(path.join(config.storageDir, INGESTION_JOURNAL_FILENAME)) stream.setEncoding("utf8") let buffered = "" @@ -578,17 +572,7 @@ async function readIngestionFileSnapshot( ): Promise<{ values: IngestionFileState[]; byPath: Map } | null> { const values: IngestionFileState[] = [] const byPath = new Map() - const stream = createReadStream(path.join(config.storageDir, header.fileSnapshot)) - stream.setEncoding("utf8") - let buffered = "" - - const applyLine = (line: string): boolean => { - let value: unknown - try { - value = JSON.parse(line) as unknown - } catch { - return false - } + const accept = (value: unknown): boolean => { if (!isIngestionFileState(value, header.policyFingerprint) || byPath.has(value.relativePath)) { return false } @@ -598,22 +582,11 @@ async function readIngestionFileSnapshot( } try { - for await (const chunk of stream) { - buffered += typeof chunk === "string" ? chunk : chunk.toString("utf8") - let lineEnd = buffered.indexOf("\n") - while (lineEnd >= 0) { - const line = buffered.slice(0, lineEnd) - buffered = buffered.slice(lineEnd + 1) - if (!line || !applyLine(line)) { - return null - } - lineEnd = buffered.indexOf("\n") - } - } - if (buffered && !applyLine(buffered)) { - return null - } - return values.length === header.fileCount ? { values, byPath } : null + const complete = await acceptJsonLines( + path.join(config.storageDir, header.fileSnapshot), + accept, + ) + return complete && values.length === header.fileCount ? { values, byPath } : null } catch (error) { if (isNodeError(error) && error.code === "ENOENT") { return null @@ -627,19 +600,8 @@ function isIngestionRunState(value: unknown, config: Config): value is Ingestion !isRecord(value) || (value.version !== LEGACY_INGESTION_STATE_VERSION && value.version !== INGESTION_STATE_VERSION) || - !isUuidV4(value.runId) || - (value.mode !== "incremental" && value.mode !== "rebuild") || - !isIngestionRunStatus(value.status) || - typeof value.tableName !== "string" || - (value.previousTableName !== null && typeof value.previousTableName !== "string") || - typeof value.policyFingerprint !== "string" || - value.policyFingerprint.length === 0 || - !isPositiveSafeInteger(value.batchSize) || - !isIsoTimestamp(value.createdAt) || - !isIsoTimestamp(value.updatedAt) || - !isIsoTimestamp(value.lastActivityAt) || - typeof value.resumed !== "boolean" || - !Array.isArray(value.files) + !Array.isArray(value.files) || + !isIngestionRunMetadata(value, config) ) { return false } @@ -648,22 +610,7 @@ function isIngestionRunState(value: unknown, config: Config): value is Ingestion if (!value.files.every((file) => isIngestionFileState(file, policyFingerprint))) { return false } - - const uniquePaths = new Set(value.files.map((file) => file.relativePath)) - if (uniquePaths.size !== value.files.length || !isManagedTableName(value.tableName, config)) { - return false - } - - if (value.mode === "incremental") { - return value.previousTableName === null - } - - return ( - value.tableName === generationTableName(config.tableName, value.runId) && - value.previousTableName !== null && - value.previousTableName !== value.tableName && - isManagedTableName(value.previousTableName, config) - ) + return new Set(value.files.map((file) => file.relativePath)).size === value.files.length } function isPersistedIngestionRunHeader( diff --git a/packages/ragmir-core/src/json-lines.ts b/packages/ragmir-core/src/json-lines.ts new file mode 100644 index 0000000..12670fb --- /dev/null +++ b/packages/ragmir-core/src/json-lines.ts @@ -0,0 +1,37 @@ +import { createReadStream } from "node:fs" + +/** + * Read a JSON Lines file and hand each parsed line to `accept`. Returns false as soon as a line is + * empty, not JSON, or rejected, so a truncated or tampered file is never half trusted. Errors + * opening or reading the file, ENOENT included, reach the caller. + */ +export async function acceptJsonLines( + filePath: string, + accept: (value: unknown) => boolean, +): Promise { + const acceptLine = (line: string): boolean => { + let value: unknown + try { + value = JSON.parse(line) as unknown + } catch { + return false + } + return accept(value) + } + const stream = createReadStream(filePath) + stream.setEncoding("utf8") + let buffered = "" + for await (const chunk of stream) { + buffered += typeof chunk === "string" ? chunk : chunk.toString("utf8") + let lineEnd = buffered.indexOf("\n") + while (lineEnd >= 0) { + const line = buffered.slice(0, lineEnd) + buffered = buffered.slice(lineEnd + 1) + if (!line || !acceptLine(line)) { + return false + } + lineEnd = buffered.indexOf("\n") + } + } + return !buffered || acceptLine(buffered) +} diff --git a/packages/ragmir-core/src/ocr-cache.ts b/packages/ragmir-core/src/ocr-cache.ts index 92b934b..6366caf 100644 --- a/packages/ragmir-core/src/ocr-cache.ts +++ b/packages/ragmir-core/src/ocr-cache.ts @@ -27,7 +27,7 @@ interface PdfOcrCacheRecord extends PdfOcrCacheIdentity { text: string } -export function pdfOcrCacheKey(identity: PdfOcrCacheIdentity): string { +function pdfOcrCacheKey(identity: PdfOcrCacheIdentity): string { return createHash("sha256").update(JSON.stringify(identity)).digest("hex") } diff --git a/packages/ragmir-core/src/ocr.ts b/packages/ragmir-core/src/ocr.ts index f99ff7b..b9eee11 100644 --- a/packages/ragmir-core/src/ocr.ts +++ b/packages/ragmir-core/src/ocr.ts @@ -15,7 +15,7 @@ import { mutateProjectConfig } from "./project-config-file.js" const OCR_COMMAND_TIMEOUT_MS = 120_000 const OCR_PROBE_TIMEOUT_MS = 10_000 const OCR_PROCESS_KILL_GRACE_MS = 1_000 -export const OCR_RENDER_DPI = 300 +const OCR_RENDER_DPI = 300 /** Longest rendered page side in pixels: US Legal at 300 DPI, so standard paper keeps 300 DPI. */ const MAX_OCR_RENDER_PIXELS = 4_200 export const MAX_OCR_BATCH_PAGES = 16 diff --git a/packages/ragmir-core/src/package-manager.ts b/packages/ragmir-core/src/package-manager.ts index 870d4e7..c410f4a 100644 --- a/packages/ragmir-core/src/package-manager.ts +++ b/packages/ragmir-core/src/package-manager.ts @@ -66,7 +66,9 @@ function displayRunnerCommand(root: string, runnerPath: string, args: string[]): return `node ${suffix}` } +/** @deprecated Use `rgrCommand`. @alias */ export const ragmirCommand = rgrCommand +/** @deprecated Use `rgrCommand`. @alias */ export const kbCommand = rgrCommand async function packageJsonManager(root: string): Promise { diff --git a/packages/ragmir-core/src/quality-report.ts b/packages/ragmir-core/src/quality-report.ts index 75c8303..7d75687 100644 --- a/packages/ragmir-core/src/quality-report.ts +++ b/packages/ragmir-core/src/quality-report.ts @@ -54,7 +54,7 @@ export function summarizeIndexedCorpus(files: Iterable): { return { fileCount, chunkCount, corpusFingerprint: digest.digest("hex") } } -export async function fingerprintFile(filePath: string, signal?: AbortSignal): Promise { +async function fingerprintFile(filePath: string, signal?: AbortSignal): Promise { return sha256(await readFile(filePath, signal ? { signal } : undefined)) } diff --git a/packages/ragmir-core/src/query-sanitizer.ts b/packages/ragmir-core/src/query-sanitizer.ts index 8d29144..0b2f411 100644 --- a/packages/ragmir-core/src/query-sanitizer.ts +++ b/packages/ragmir-core/src/query-sanitizer.ts @@ -1,6 +1,6 @@ import { RagmirError } from "./errors.js" -export const MAX_RETRIEVAL_QUERY_LENGTH = 20_000 +const MAX_RETRIEVAL_QUERY_LENGTH = 20_000 export interface SanitizedQuery { query: string diff --git a/packages/ragmir-core/src/skill.ts b/packages/ragmir-core/src/skill.ts index b134520..5167480 100644 --- a/packages/ragmir-core/src/skill.ts +++ b/packages/ragmir-core/src/skill.ts @@ -96,7 +96,7 @@ const MCP_SERVER_NAME_PATTERN = /^[A-Za-z0-9_-]+$/u const MANAGED_SKILL_METADATA_FILENAME = ".ragmir-managed.json" export const SKILL_NAMES = [PRIMARY_SKILL_NAME] as const /** Skills of retired commands (audio, reports, legal dossiers); their instructions now fail. */ -export const RETIRED_SKILL_NAMES = [ +const RETIRED_SKILL_NAMES = [ "ragmir-audio-summary", "ragmir-legal-dossier", "ragmir-markdown-report", diff --git a/packages/ragmir-core/src/source-fingerprint-cache.ts b/packages/ragmir-core/src/source-fingerprint-cache.ts index abcec66..307d916 100644 --- a/packages/ragmir-core/src/source-fingerprint-cache.ts +++ b/packages/ragmir-core/src/source-fingerprint-cache.ts @@ -1,9 +1,9 @@ import { randomUUID } from "node:crypto" -import { createReadStream } from "node:fs" import { open as openFile, rm } from "node:fs/promises" import path from "node:path" import { renameWithRetry } from "./fs-retry.js" import { isRecord } from "./guards.js" +import { acceptJsonLines } from "./json-lines.js" import { ensurePrivateDirectory, hardenPrivateFile } from "./permissions.js" import type { Config, SourceFingerprintMode } from "./types.js" @@ -31,18 +31,9 @@ export async function readSourceFingerprintCache( config: Config, ): Promise | null> { const records = new Map() - const stream = createReadStream(path.join(config.storageDir, SOURCE_FINGERPRINT_CACHE_FILENAME)) - stream.setEncoding("utf8") - let buffered = "" let headerRead = false - const applyLine = (line: string): boolean => { - let value: unknown - try { - value = JSON.parse(line) as unknown - } catch { - return false - } + const accept = (value: unknown): boolean => { if (!headerRead) { headerRead = true return isRecord(value) && value.version === CACHE_VERSION && value.type === "header" @@ -55,22 +46,11 @@ export async function readSourceFingerprintCache( } try { - for await (const chunk of stream) { - buffered += typeof chunk === "string" ? chunk : chunk.toString("utf8") - let lineEnd = buffered.indexOf("\n") - while (lineEnd >= 0) { - const line = buffered.slice(0, lineEnd) - buffered = buffered.slice(lineEnd + 1) - if (!line || !applyLine(line)) { - return null - } - lineEnd = buffered.indexOf("\n") - } - } - if (buffered && !applyLine(buffered)) { - return null - } - return headerRead ? records : null + const complete = await acceptJsonLines( + path.join(config.storageDir, SOURCE_FINGERPRINT_CACHE_FILENAME), + accept, + ) + return complete && headerRead ? records : null } catch (error) { if (isNodeError(error) && error.code === "ENOENT") { return new Map() diff --git a/packages/ragmir-core/src/source-weights.ts b/packages/ragmir-core/src/source-weights.ts index 5dc2c3b..3d076a3 100644 --- a/packages/ragmir-core/src/source-weights.ts +++ b/packages/ragmir-core/src/source-weights.ts @@ -9,7 +9,7 @@ import type { Config } from "./types.js" * `pathPriors: false` turns them off and `sourceWeights` can override any pattern. Agent * instruction files stay neutral: in many projects they hold the rules a question is about. */ -export const DEFAULT_PATH_PRIORS: Readonly> = { +const DEFAULT_PATH_PRIORS: Readonly> = { "**/__tests__/**": 0.85, "**/*.{test,spec}.*": 0.85, "**/test/**": 0.85, diff --git a/packages/ragmir-core/src/statistics.ts b/packages/ragmir-core/src/statistics.ts new file mode 100644 index 0000000..d509700 --- /dev/null +++ b/packages/ragmir-core/src/statistics.ts @@ -0,0 +1,7 @@ +/** + * Nearest-rank percentile of ascending values: the smallest value with at least `quantile` of the + * values at or below it. Empty input gives 0. + */ +export function percentile(sortedValues: readonly number[], quantile: number): number { + return sortedValues[Math.max(0, Math.ceil(quantile * sortedValues.length) - 1)] ?? 0 +} diff --git a/packages/ragmir-core/src/test-support/timeouts.ts b/packages/ragmir-core/src/test-support/timeouts.ts index 2df8ff1..2500af9 100644 --- a/packages/ragmir-core/src/test-support/timeouts.ts +++ b/packages/ragmir-core/src/test-support/timeouts.ts @@ -2,7 +2,7 @@ // git costs 16 times more there, an fsync 9 times more, and creating and removing small files 9 // times more, so ingesting the demo corpus takes 409 ms against 147 ms on Linux. Test timeouts are // calibrated on Linux and macOS, then scaled on Windows; vitest.config.ts scales the defaults. -export const PLATFORM_TIMEOUT_FACTOR = process.platform === "win32" ? 3 : 1 +const PLATFORM_TIMEOUT_FACTOR = process.platform === "win32" ? 3 : 1 /** Timeout for an integration test that ingests, rebuilds, or migrates an index. */ export function integrationTimeout(milliseconds: number): number { diff --git a/packages/ragmir-core/src/vector-index.ts b/packages/ragmir-core/src/vector-index.ts index b8fdd34..b089757 100644 --- a/packages/ragmir-core/src/vector-index.ts +++ b/packages/ragmir-core/src/vector-index.ts @@ -9,10 +9,10 @@ import type { VectorIndexStrategy, } from "./types.js" -export const VECTOR_INDEX_NAME = "vector_idx" -export const RELATIVE_PATH_INDEX_NAME = "relativePath_idx" +const VECTOR_INDEX_NAME = "vector_idx" +const RELATIVE_PATH_INDEX_NAME = "relativePath_idx" export const ANN_MINIMUM_ROWS = 100_000 -export const SCALAR_INDEX_MINIMUM_ROWS = 10_000 +const SCALAR_INDEX_MINIMUM_ROWS = 10_000 const VECTOR_INDEX_POLICY_VERSION = 1 const L_CORPUS_ROWS = 1_000_000 diff --git a/packages/ragmir-core/src/workload.ts b/packages/ragmir-core/src/workload.ts index 8d10196..a08248a 100644 --- a/packages/ragmir-core/src/workload.ts +++ b/packages/ragmir-core/src/workload.ts @@ -4,7 +4,7 @@ import { RagmirError } from "./errors.js" import { throwIfAborted } from "./operation.js" import type { Config, WorkloadKind, WorkloadLimit } from "./types.js" -export const WORKLOAD_DIAGNOSTICS_CHANNEL = "ragmir:workload" +const WORKLOAD_DIAGNOSTICS_CHANNEL = "ragmir:workload" const workloadDiagnostics = channel(WORKLOAD_DIAGNOSTICS_CHANNEL) export interface WorkloadAdmission { diff --git a/packages/ragmir-landing/src/components/error-page.astro b/packages/ragmir-landing/src/components/error-page.astro new file mode 100644 index 0000000..fbfe5d4 --- /dev/null +++ b/packages/ragmir-landing/src/components/error-page.astro @@ -0,0 +1,71 @@ +--- +import { defaultLang, getLocalizedUrl } from "../i18n/utils" +import Layout from "../layouts/layout.astro" +import { RagmirBackground } from "./ragmir-background" +import RagmirLogo from "./ragmir-logo.astro" + +interface ErrorCopy { + title: string + metaDescription: string + message: string +} + +interface Props extends ErrorCopy { + code: string + /** The host serves one error page for every locale, so French is applied from the URL path. */ + french: ErrorCopy +} + +const { code, title, metaDescription, message, french } = Astro.props +const homeUrl = getLocalizedUrl("/", defaultLang) +--- + + + + +
+ + + + +

{code}

+

{message}

+ + + Back to Ragmir + +
+ + + diff --git a/packages/ragmir-landing/src/components/ui/tabs.tsx b/packages/ragmir-landing/src/components/ui/tabs.tsx index ef88710..2d15e2e 100644 --- a/packages/ragmir-landing/src/components/ui/tabs.tsx +++ b/packages/ragmir-landing/src/components/ui/tabs.tsx @@ -76,4 +76,4 @@ function TabsContent({ className, ...props }: React.ComponentProps - - -
- - - Ragmir - - - -

404

-

- The page you are looking for does not exist or has moved. -

- - - Back to Ragmir - -
- - {/* Localize French client-side from the URL path (/fr/...). */} - - + metaDescription="The page you are looking for does not exist or has moved." + message="The page you are looking for does not exist or has moved." + french={{ + title: "Page introuvable | Ragmir", + metaDescription: "Cette page n'existe pas ou a été déplacée.", + message: "Cette page n'existe pas ou a été déplacée.", + }} +/> diff --git a/packages/ragmir-landing/src/pages/500.astro b/packages/ragmir-landing/src/pages/500.astro index 966b02e..dedd3e0 100644 --- a/packages/ragmir-landing/src/pages/500.astro +++ b/packages/ragmir-landing/src/pages/500.astro @@ -1,65 +1,15 @@ --- -import { RagmirBackground } from "../components/ragmir-background" -import { defaultLang, getLocalizedUrl } from "../i18n/utils" -import Layout from "../layouts/layout.astro" - -const homeUrl = getLocalizedUrl("/", defaultLang) +import ErrorPage from "../components/error-page.astro" --- - - - -
- - - Ragmir - - - -

500

-

- Something went wrong on our side. Please try again. -

- - - Back to Ragmir - -
- - {/* Localize French client-side from the URL path (/fr/...). */} - - + metaDescription="Something went wrong on our side." + message="Something went wrong on our side. Please try again." + french={{ + title: "Erreur serveur | Ragmir", + metaDescription: "Une erreur s'est produite de notre côté.", + message: "Une erreur s'est produite de notre côté. Veuillez réessayer.", + }} +/> From 897089823e35f44d5970f396b24b5fe574d36688 Mon Sep 17 00:00:00 2001 From: Jean-Baptiste THERY Date: Sun, 1 Nov 2026 19:17:54 +0700 Subject: [PATCH 3/5] feat(bench): compare Ragmir with LangChain.js, Orama, and MiniSearch and gate on Knip - add `pnpm bench:competitors`: the same tracked files, questions, embedding model, and vectors for every stack, with off-topic and near-domain refusals and latency - add the public question fixture: 99 developer questions with every answering file, 30 off-topic and 30 near-domain questions - add Knip to `pnpm validate` with entries for benchmark workers and release scripts, declare TypeScript at the root for the API check, and document the rule for contributors --- AGENTS.md | 1 + CLAUDE.md | 1 + CONTRIBUTING.md | 10 +- knip.json | 21 + package.json | 6 +- packages/ragmir-core/benchmarks/README.md | 9 + .../ragmir-core/benchmarks/competitors.mjs | 310 +++++ .../fixtures/repository-questions.json | 1172 +++++++++++++++++ packages/ragmir-core/package.json | 5 + pnpm-lock.yaml | 673 +++++++++- 10 files changed, 2201 insertions(+), 7 deletions(-) create mode 100644 knip.json create mode 100644 packages/ragmir-core/benchmarks/competitors.mjs create mode 100644 packages/ragmir-core/benchmarks/fixtures/repository-questions.json diff --git a/AGENTS.md b/AGENTS.md index f9c6350..0b1caf7 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -59,6 +59,7 @@ ## Validation - Run the smallest relevant check while editing and `pnpm validate` before a release pull request. +- Keep `pnpm knip` clean: delete unused files, exports, and dependencies instead of ignoring them. - Reconcile `git status` with the intended scope before staging. Never stage secrets or generated local state. - Use Conventional Commits. Commit and push only when the user authorizes them. - Every commit promoted to `main` that can trigger semantic-release must include these exact body diff --git a/CLAUDE.md b/CLAUDE.md index 30564c7..53503cb 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -5,6 +5,7 @@ ```bash pnpm bootstrap pnpm validate +pnpm knip pnpm dev:landing pnpm example ``` diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 4245a0c..7717477 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -50,9 +50,13 @@ Before opening a pull request, run: pnpm validate ``` -`pnpm validate` runs Biome, a dependency security audit, TypeScript checks, coverage-gated Vitest, -production CLI and MCP smoke tests, the static landing build, public API checks, and npm package and -release-artifact checks. +`pnpm validate` runs Biome, Knip, a dependency security audit, TypeScript checks, coverage-gated +Vitest, production CLI and MCP smoke tests, the static landing build, public API checks, and npm +package and release-artifact checks. + +`pnpm knip` reports unused files, exports, dependencies, and duplicate exports across the workspace. +Remove what it finds rather than ignoring it. Tag an export `@public` only when a benchmark or +script imports it from the build, and `@alias` only for a deliberate public alias. To smoke-test the library API against your local build while developing Ragmir Core, run `pnpm example` (see diff --git a/knip.json b/knip.json new file mode 100644 index 0000000..b9f8e29 --- /dev/null +++ b/knip.json @@ -0,0 +1,21 @@ +{ + "$schema": "https://unpkg.com/knip@6/schema.json", + "ignoreExportsUsedInFile": { + "interface": true, + "type": true + }, + "ignoreBinaries": ["mise", "textutil"], + "workspaces": { + ".": { + "entry": ["scripts/*.mjs", "tests/public-api-consumer/consumer.ts"], + "project": ["scripts/**/*.mjs", "tests/**/*.ts"] + }, + "packages/ragmir-core": { + "entry": ["benchmarks/*.mjs", "scripts/*.mjs", "examples/**/*.{mjs,ts}"], + "project": ["src/**/*.ts", "benchmarks/**/*.mjs", "scripts/**/*.mjs"] + }, + "packages/ragmir-landing": { + "ignoreDependencies": ["@astrojs/check"] + } + } +} diff --git a/package.json b/package.json index 654aaa8..cdf721d 100644 --- a/package.json +++ b/package.json @@ -34,6 +34,7 @@ "api:check": "tsc -p tests/public-api-consumer/tsconfig.json", "bootstrap": "mise install && pnpm install", "bench:compare": "pnpm --filter @jcode.labs/ragmir bench:compare", + "bench:competitors": "pnpm --filter @jcode.labs/ragmir bench:competitors", "bench:cli-startup": "pnpm --filter @jcode.labs/ragmir bench:cli-startup", "bench:discovery": "pnpm --filter @jcode.labs/ragmir bench:discovery", "bench:ingest-memory": "pnpm --filter @jcode.labs/ragmir bench:ingest-memory", @@ -61,6 +62,7 @@ "example": "pnpm --filter @jcode.labs/ragmir build && node packages/ragmir-core/examples/library-api-demo/run.mjs", "format": "biome format --write .", "lint": "biome ci .", + "knip": "knip --no-progress", "lint:fix": "biome check --write .", "license:check": "node scripts/license-policy-smoke.mjs", "offline:smoke": "node scripts/offline-install-smoke.mjs", @@ -72,7 +74,7 @@ "smoke": "pnpm public:smoke && node scripts/smoke.mjs && pnpm --filter @jcode.labs/ragmir mcp:smoke", "test": "pnpm --filter @jcode.labs/ragmir-landing test && pnpm --filter @jcode.labs/ragmir test", "test:coverage": "pnpm --filter @jcode.labs/ragmir-landing test:coverage && pnpm --filter @jcode.labs/ragmir test:coverage", - "validate": "pnpm lint && pnpm audit:security && pnpm check && pnpm test:coverage && pnpm build && pnpm api:check && pnpm smoke && pnpm package:check && pnpm release:semantic:smoke && pnpm release:artifacts", + "validate": "pnpm lint && pnpm knip && pnpm audit:security && pnpm check && pnpm test:coverage && pnpm build && pnpm api:check && pnpm smoke && pnpm package:check && pnpm release:semantic:smoke && pnpm release:artifacts", "bench:developer-evidence": "pnpm --filter @jcode.labs/ragmir bench:developer-evidence" }, "devDependencies": { @@ -81,7 +83,9 @@ "@commitlint/config-conventional": "^21.2.4", "@semantic-release/exec": "^7.1.0", "@vitest/coverage-v8": "5.0.5", + "knip": "^6.44.1", "semantic-release": "^25.0.9", + "typescript": "^7.0.2", "yaml": "^2.9.1" } } diff --git a/packages/ragmir-core/benchmarks/README.md b/packages/ragmir-core/benchmarks/README.md index 9609c9c..7622a68 100644 --- a/packages/ragmir-core/benchmarks/README.md +++ b/packages/ragmir-core/benchmarks/README.md @@ -17,6 +17,7 @@ Generated corpora, model caches, and raw results stay outside Git. | `pnpm bench:observability` | Diagnostic completeness, privacy, and disabled-path cost | | `pnpm bench:parsers [-- --stress]` | Office, EPUB, PDF throughput, memory, citations, and malformed input | | `pnpm bench:compare -- --baseline baseline.json --current current.json` | Comparable result regression | +| `pnpm bench:competitors [-- --model-path /path/to/models]` | Ragmir against LangChain.js, Orama, and MiniSearch on this repository | Additional root scripts cover CLI startup, discovery, ingestion memory, metadata, local-hash runtime, storage maintenance, status, runtime reuse, OCR cache, generation retention, concurrency, @@ -35,6 +36,14 @@ and explicit reranker, compression, hashing, and content-dedup experiments. complete corpus independently, with a one-chunk document cap and no calibrated abstention. The gate covers exact references, long-query preservation, and spreadsheet header retrieval. Other outcomes are reported, not presented as representative developer-task success rates. +- `bench:competitors` indexes the tracked documentation and source of this repository at the + current commit and asks the questions in `fixtures/repository-questions.json`: 99 developer + questions with every file that answers them, 30 off-topic questions, and 30 near-domain + questions about features Ragmir does not have. The other stacks read the same files split by + LangChain's `RecursiveCharacterTextSplitter` (1,000 characters, 200 overlap), and every vector + side uses Ragmir's embedding function, so models, prefixes, and vectors are identical. Orama + runs with its default 0.8 hybrid similarity and with 0.1, which lets its vector side rank. A + question counts as answered when one of its files appears; a refusal is an empty result. - `bench:compare` recognizes scale and quality reports separately. It validates both quality runs, their absolute gates, and explicit workload versions. Missing identities or metrics are invalid, incompatible workloads are inconclusive, and only complete comparable reports can pass. diff --git a/packages/ragmir-core/benchmarks/competitors.mjs b/packages/ragmir-core/benchmarks/competitors.mjs new file mode 100644 index 0000000..b312ccb --- /dev/null +++ b/packages/ragmir-core/benchmarks/competitors.mjs @@ -0,0 +1,310 @@ +// Compare Ragmir with popular local TypeScript retrieval stacks on the public Ragmir repository: +// the same tracked files, the same questions, and, for every vector side, the same embedding +// model and vectors. Writes benchmarks/.results/competitors.json and prints a Markdown table. +import { execFileSync } from "node:child_process" +import { mkdir, mkdtemp, readFile, rm, writeFile } from "node:fs/promises" +import os from "node:os" +import path from "node:path" +import { performance } from "node:perf_hooks" +import { fileURLToPath } from "node:url" +import { RecursiveCharacterTextSplitter } from "@langchain/textsplitters" +import { create, insertMultiple, search as searchOrama } from "@orama/orama" +import MiniSearch from "minisearch" +import picomatch from "picomatch" +import { DEFAULT_CONFIG } from "../dist/defaults.js" +import { embedTexts } from "../dist/embeddings.js" +import { createRagmirClient, initProject, loadConfig } from "../dist/index.js" +import { environmentMetadata } from "./lib/metrics.mjs" + +const packageRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), "..") +const repositoryRoot = path.resolve(packageRoot, "../..") +const options = parseArguments(process.argv.slice(2)) +const model = options.model ?? "mixedbread-ai/mxbai-embed-xsmall-v1" +const revision = options.revision ?? "e6ac24e5d6efb8782b59de1647b3ececb4ece94e" +const modelPath = path.resolve(options.modelPath ?? path.join(repositoryRoot, ".ragmir/models")) +const resultPath = path.resolve( + options.result ?? path.join(packageRoot, "benchmarks/.results/competitors.json"), +) +const TOP_K = 10 +// The documentation and source a developer agent asks about, tracked in Git. +const SOURCES = [ + "README.md", + "AGENTS.md", + "CLAUDE.md", + "CHANGELOG.md", + "CONTRIBUTING.md", + "RELEASING.md", + "SECURITY.md", + "SECURITY-HARDENING.md", + "llms.txt", + "package.json", + "docs/**/*.md", + "packages/*/README.md", + "packages/*/package.json", + "packages/ragmir-core/src/**/*.ts", + "packages/ragmir-core/skills/**/*.md", + "packages/ragmir-core/benchmarks/README.md", + "packages/ragmir-landing/src/**/*.{astro,ts,tsx}", + "packages/ragmir-landing/messages/*.json", + "scripts/*.mjs", +] + +const questions = JSON.parse( + await readFile(path.join(packageRoot, "benchmarks/fixtures/repository-questions.json"), "utf8"), +) +const isSource = picomatch(SOURCES, { dot: true }) +const files = execFileSync("git", ["ls-files", "-z"], { cwd: repositoryRoot, encoding: "utf8" }) + .split("\0") + .filter((file) => file && isSource(file)) + .sort() +const commit = execFileSync("git", ["rev-parse", "HEAD"], { cwd: repositoryRoot, encoding: "utf8" }).trim() + +const workspace = await mkdtemp(path.join(os.tmpdir(), "ragmir-competitors-")) +const systems = [] +try { + for (const file of files) { + const target = path.join(workspace, file) + await mkdir(path.dirname(target), { recursive: true }) + await writeFile(target, await readFile(path.join(repositoryRoot, file))) + } + await initProject(workspace) + const semanticConfig = { + ...DEFAULT_CONFIG, + sources: SOURCES, + embeddingProvider: "transformers", + embeddingModel: model, + embeddingModelRevision: revision, + embeddingModelPath: modelPath, + retrievalProfile: "quality", + } + + for (const provider of ["transformers", "local-hash"]) { + const config = + provider === "transformers" + ? semanticConfig + : { ...DEFAULT_CONFIG, sources: SOURCES, retrievalProfile: "quality" } + await writeFile(path.join(workspace, ".ragmir/config.json"), JSON.stringify(config)) + const client = await createRagmirClient({ cwd: workspace }) + try { + const startedAt = performance.now() + const ingest = await client.ingest({ rebuild: true }) + if (ingest.errors.length > 0) throw new Error(JSON.stringify(ingest.errors)) + const indexMs = performance.now() - startedAt + systems.push( + await measure( + provider === "transformers" + ? `Ragmir (hybrid, ${shortModel(model)})` + : "Ragmir (local-hash, no model)", + indexMs, + async (query) => + (await client.search(query, { topK: TOP_K })).map((result) => result.relativePath), + ), + ) + } finally { + await client.close() + } + } + + // Every other stack reads the same files, split by LangChain's recommended splitter, and the + // vector stacks reuse Ragmir's embedding function: identical model, prefixes, and vectors. + await writeFile(path.join(workspace, ".ragmir/config.json"), JSON.stringify(semanticConfig)) + const embeddingConfig = await loadConfig(workspace) + const splitter = new RecursiveCharacterTextSplitter({ chunkSize: 1000, chunkOverlap: 200 }) + const splitStartedAt = performance.now() + const chunks = [] + for (const file of files) { + const text = await readFile(path.join(repositoryRoot, file), "utf8") + for (const piece of await splitter.splitText(text)) { + chunks.push({ id: String(chunks.length), path: file, text: piece }) + } + } + const splitMs = performance.now() - splitStartedAt + const embedStartedAt = performance.now() + const vectors = [] + for (let offset = 0; offset < chunks.length; offset += 64) { + vectors.push( + ...(await embedTexts( + chunks.slice(offset, offset + 64).map((chunk) => chunk.text), + embeddingConfig, + "document", + )), + ) + } + const embedMs = performance.now() - embedStartedAt + const embedQuery = async (query) => (await embedTexts([query], embeddingConfig, "query"))[0] + + // LangChain's MemoryVectorStore ranks by exact cosine similarity over every stored vector. + systems.push( + await measure("LangChain.js splitter + MemoryVectorStore-style cosine", splitMs + embedMs, async (query) => { + const queryVector = await embedQuery(query) + return vectors + .map((vector, index) => ({ index, score: cosine(queryVector, vector) })) + .sort((left, right) => right.score - left.score) + .slice(0, TOP_K) + .map(({ index }) => chunks[index].path) + }), + ) + + const oramaStartedAt = performance.now() + const orama = create({ + schema: { path: "string", text: "string", embedding: `vector[${vectors[0].length}]` }, + }) + await insertMultiple( + orama, + chunks.map((chunk, index) => ({ path: chunk.path, text: chunk.text, embedding: vectors[index] })), + ) + const oramaMs = splitMs + embedMs + (performance.now() - oramaStartedAt) + // Orama's default 0.8 similarity drops most vector hits of this model, so both its default and + // a 0.1 threshold that lets the vector side rank are reported. + for (const similarity of [0.8, 0.1]) { + systems.push( + await measure(`Orama hybrid (similarity ${similarity})`, oramaMs, async (query) => { + const result = await searchOrama(orama, { + mode: "hybrid", + term: query, + vector: { value: await embedQuery(query), property: "embedding" }, + similarity, + limit: TOP_K, + }) + return result.hits.map((hit) => hit.document.path) + }), + ) + } + + const miniStartedAt = performance.now() + const minisearch = new MiniSearch({ fields: ["text"], storeFields: ["path"] }) + minisearch.addAll(chunks) + const miniMs = splitMs + (performance.now() - miniStartedAt) + systems.push( + await measure("MiniSearch BM25", miniMs, async (query) => + minisearch + .search(query) + .slice(0, TOP_K) + .map((hit) => hit.path), + ), + ) + + const report = { + schemaVersion: 1, + createdAt: new Date().toISOString(), + commit, + environment: await environmentMetadata(), + corpus: { trackedFiles: files.length, langchainChunks: chunks.length }, + model: { name: model, revision }, + questions: { + answerable: questions.answerable.length, + unrelated: questions.unrelated.length, + nearDomain: questions.nearDomain.length, + }, + systems, + } + await mkdir(path.dirname(resultPath), { recursive: true }) + await writeFile(resultPath, `${JSON.stringify(report, null, 2)}\n`) + console.log(markdownTable(report)) + console.log(`\nWrote ${path.relative(process.cwd(), resultPath)}`) +} finally { + await rm(workspace, { recursive: true, force: true }) +} + +async function measure(name, indexMs, retrieve) { + for (const entry of questions.answerable.slice(0, 3)) await retrieve(entry.query) + const latencies = [] + const timed = async (query) => { + const startedAt = performance.now() + const paths = await retrieve(query) + latencies.push(performance.now() - startedAt) + return paths + } + const ranking = [] + for (const entry of questions.answerable) { + ranking.push(rankingMetrics(await timed(entry.query), entry.expectedPaths)) + } + const refused = async (list) => { + let count = 0 + for (const query of list) count += (await timed(query)).length === 0 ? 1 : 0 + return count + } + const unrelatedRefused = await refused(questions.unrelated) + const nearDomainRefused = await refused(questions.nearDomain) + latencies.sort((left, right) => left - right) + const share = (key) => ranking.filter((entry) => entry[key]).length / ranking.length + const mean = (key) => ranking.reduce((sum, entry) => sum + entry[key], 0) / ranking.length + return { + name, + indexSeconds: round(indexMs / 1000, 1), + hitAt1: round(share("hit1")), + hitAt3: round(share("hit3")), + hitAt10: round(share("hit10")), + mrrAt10: round(mean("rr")), + ndcgAt10: round(mean("ndcg")), + unrelatedRefused, + nearDomainRefused, + medianMs: round(latencies[Math.floor(latencies.length / 2)], 1), + p95Ms: round(latencies[Math.floor(latencies.length * 0.95)], 1), + } +} + +/** File-level metrics: a passage counts when its file answers the question; repeats count once. */ +function rankingMetrics(paths, expectedPaths) { + const expected = new Set(expectedPaths) + const seen = new Set() + let dcg = 0 + let reciprocalRank = 0 + for (const [index, file] of paths.slice(0, TOP_K).entries()) { + if (!expected.has(file) || seen.has(file)) continue + seen.add(file) + dcg += 1 / Math.log2(index + 2) + if (reciprocalRank === 0) reciprocalRank = 1 / (index + 1) + } + let ideal = 0 + for (let index = 0; index < Math.min(expected.size, TOP_K); index += 1) ideal += 1 / Math.log2(index + 2) + const hit = (k) => paths.slice(0, k).some((file) => expected.has(file)) + return { hit1: hit(1), hit3: hit(3), hit10: hit(TOP_K), rr: reciprocalRank, ndcg: dcg / ideal } +} + +function cosine(left, right) { + let dot = 0 + let leftNorm = 0 + let rightNorm = 0 + for (let index = 0; index < left.length; index += 1) { + dot += left[index] * right[index] + leftNorm += left[index] * left[index] + rightNorm += right[index] * right[index] + } + return dot / Math.sqrt(leftNorm * rightNorm) +} + +function markdownTable(report) { + const percent = (value) => `${Math.round(value * 100)}%` + const rows = report.systems.map( + (system) => + `| ${system.name} | ${percent(system.hitAt3)} | ${percent(system.hitAt10)} | ${system.mrrAt10} | ${system.unrelatedRefused}/${report.questions.unrelated} | ${system.nearDomainRefused}/${report.questions.nearDomain} | ${system.medianMs} ms |`, + ) + return [ + `${report.questions.answerable} answerable questions over ${report.corpus.trackedFiles} tracked files at ${report.commit.slice(0, 7)}`, + "", + "| System | Answer in top 3 | Answer in top 10 | MRR@10 | Off-topic refused | Near-domain refused | Median search |", + "| --- | ---: | ---: | ---: | ---: | ---: | ---: |", + ...rows, + ].join("\n") +} + +function shortModel(name) { + return name.split("/").pop() +} + +function round(value, digits = 3) { + return Number(value.toFixed(digits)) +} + +function parseArguments(argumentsList) { + const parsed = {} + for (let index = 0; index < argumentsList.length; index += 1) { + const current = argumentsList[index] + if (!current.startsWith("--")) continue + const key = current.slice(2).replace(/-([a-z])/gu, (_, letter) => letter.toUpperCase()) + parsed[key] = argumentsList[index + 1] + index += 1 + } + return parsed +} diff --git a/packages/ragmir-core/benchmarks/fixtures/repository-questions.json b/packages/ragmir-core/benchmarks/fixtures/repository-questions.json new file mode 100644 index 0000000..bc743ad --- /dev/null +++ b/packages/ragmir-core/benchmarks/fixtures/repository-questions.json @@ -0,0 +1,1172 @@ +{ + "description": "Questions about the public Ragmir repository: answerable developer questions with every file that answers them, off-topic questions, and near-domain questions about features Ragmir does not have. Expected paths are tracked files.", + "answerable": [ + { + "id": "guided-setup-1", + "query": "How should a coding agent inspect a repository before installing Ragmir?", + "expectedPaths": [ + "docs/quick-start.md", + "packages/ragmir-core/src/setup-prompt.ts", + "README.md", + "docs/agent-integration.md", + "packages/ragmir-core/README.md" + ] + }, + { + "id": "guided-setup-2", + "query": "Which approvals must the guided Ragmir setup request before changing dependencies or downloading models?", + "expectedPaths": [ + "docs/quick-start.md", + "packages/ragmir-core/src/setup-prompt.ts", + "README.md", + "docs/agent-integration.md", + "packages/ragmir-core/README.md" + ] + }, + { + "id": "guided-setup-3", + "query": "How does the guided setup choose sources and exclude secrets or generated noise?", + "expectedPaths": [ + "docs/quick-start.md", + "packages/ragmir-core/src/setup-prompt.ts", + "README.md", + "docs/agent-integration.md", + "packages/ragmir-core/README.md" + ] + }, + { + "id": "guided-setup-4", + "query": "What verification should the guided setup run after indexing the repository?", + "expectedPaths": [ + "docs/quick-start.md", + "packages/ragmir-core/src/setup-prompt.ts", + "README.md", + "docs/agent-integration.md", + "packages/ragmir-core/README.md" + ] + }, + { + "id": "guided-setup-5", + "query": "How should an existing Ragmir installation be inspected before it is upgraded?", + "expectedPaths": [ + "docs/quick-start.md", + "packages/ragmir-core/src/setup-prompt.ts", + "README.md", + "docs/agent-integration.md", + "packages/ragmir-core/README.md" + ] + }, + { + "id": "package-manager-1", + "query": "How does the guided setup prompt detect pnpm npm Yarn or Bun?", + "expectedPaths": [ + "packages/ragmir-core/src/package-manager.ts", + "packages/ragmir-core/src/setup-prompt.ts", + "docs/quick-start.md", + "packages/ragmir-landing/src/components/library-section.tsx" + ] + }, + { + "id": "package-manager-2", + "query": "Which package manager signal has priority when Ragmir builds commands?", + "expectedPaths": [ + "packages/ragmir-core/src/package-manager.ts", + "packages/ragmir-core/src/setup-prompt.ts", + "docs/quick-start.md", + "packages/ragmir-landing/src/components/library-section.tsx" + ] + }, + { + "id": "package-manager-3", + "query": "What rgr execution prefix is generated for a Bun project?", + "expectedPaths": [ + "packages/ragmir-core/src/package-manager.ts", + "packages/ragmir-core/src/setup-prompt.ts", + "docs/quick-start.md", + "packages/ragmir-landing/src/components/library-section.tsx" + ] + }, + { + "id": "package-manager-4", + "query": "How does Ragmir avoid choosing a second lockfile in a workspace?", + "expectedPaths": [ + "packages/ragmir-core/src/package-manager.ts", + "packages/ragmir-core/src/setup-prompt.ts", + "docs/quick-start.md", + "packages/ragmir-landing/src/components/library-section.tsx" + ] + }, + { + "id": "package-manager-5", + "query": "How does rgrCommand turn a detected package manager into a display command?", + "expectedPaths": [ + "packages/ragmir-core/src/package-manager.ts", + "packages/ragmir-core/src/setup-prompt.ts", + "docs/quick-start.md", + "packages/ragmir-landing/src/components/library-section.tsx" + ] + }, + { + "id": "privacy-1", + "query": "What stays local when Ragmir Core indexes and retrieves project documents?", + "expectedPaths": [ + "README.md", + "packages/ragmir-core/README.md", + "SECURITY-HARDENING.md", + "llms.txt", + "context7.json" + ] + }, + { + "id": "privacy-2", + "query": "Does Ragmir Core upload the corpus or call a language model?", + "expectedPaths": [ + "README.md", + "packages/ragmir-core/README.md", + "SECURITY-HARDENING.md", + "llms.txt", + "context7.json" + ] + }, + { + "id": "privacy-3", + "query": "Where are generated indexes reports helpers and metadata-only logs stored?", + "expectedPaths": [ + "README.md", + "packages/ragmir-core/README.md", + "SECURITY-HARDENING.md", + "llms.txt", + "context7.json" + ] + }, + { + "id": "privacy-4", + "query": "When can retrieved passages cross a network boundary?", + "expectedPaths": [ + "README.md", + "packages/ragmir-core/README.md", + "SECURITY-HARDENING.md", + "llms.txt", + "context7.json" + ] + }, + { + "id": "privacy-5", + "query": "Why is local-hash described as offline rather than semantic?", + "expectedPaths": [ + "README.md", + "packages/ragmir-core/README.md", + "SECURITY-HARDENING.md", + "llms.txt", + "context7.json" + ] + }, + { + "id": "agent-mcp-1", + "query": "How should a coding agent connect to Ragmir through the local MCP server?", + "expectedPaths": [ + "docs/agent-integration.md", + "README.md", + "docs/api-reference.md", + "llms.txt", + "context7.json", + "packages/ragmir-core/src/mcp.ts", + "packages/ragmir-core/skills/ragmir/SKILL.md" + ] + }, + { + "id": "agent-mcp-2", + "query": "Which native helpers are generated for Claude Code Codex Kimi OpenCode and Cline?", + "expectedPaths": [ + "docs/agent-integration.md", + "README.md", + "docs/api-reference.md", + "llms.txt", + "context7.json", + "packages/ragmir-core/src/mcp.ts", + "packages/ragmir-core/skills/ragmir/SKILL.md" + ] + }, + { + "id": "agent-mcp-3", + "query": "How does an MCP helper select the correct knowledge base in a monorepo?", + "expectedPaths": [ + "docs/agent-integration.md", + "README.md", + "docs/api-reference.md", + "llms.txt", + "context7.json", + "packages/ragmir-core/src/mcp.ts", + "packages/ragmir-core/skills/ragmir/SKILL.md" + ] + }, + { + "id": "agent-mcp-4", + "query": "Why should an agent read ragmir://context before retrieval?", + "expectedPaths": [ + "docs/agent-integration.md", + "README.md", + "docs/api-reference.md", + "llms.txt", + "context7.json", + "packages/ragmir-core/src/mcp.ts", + "packages/ragmir-core/skills/ragmir/SKILL.md" + ] + }, + { + "id": "agent-mcp-5", + "query": "What happens when an unmanaged same-name agent skill already exists?", + "expectedPaths": [ + "docs/agent-integration.md", + "README.md", + "docs/api-reference.md", + "llms.txt", + "context7.json", + "packages/ragmir-core/src/mcp.ts", + "packages/ragmir-core/skills/ragmir/SKILL.md" + ] + }, + { + "id": "team-1", + "query": "How do teammates synchronize Ragmir sources while keeping one local index per developer?", + "expectedPaths": [ + "docs/agent-integration.md", + "README.md", + "docs/configuration.md", + "llms.txt" + ] + }, + { + "id": "team-2", + "query": "What does a metadata-only team snapshot contain and exclude?", + "expectedPaths": [ + "docs/agent-integration.md", + "README.md", + "docs/configuration.md", + "llms.txt" + ] + }, + { + "id": "team-3", + "query": "How does team compare explain local-only peer-only and changed files?", + "expectedPaths": [ + "docs/agent-integration.md", + "README.md", + "docs/configuration.md", + "llms.txt" + ] + }, + { + "id": "team-4", + "query": "Who decides which Git Drive or folder revision is authoritative?", + "expectedPaths": [ + "docs/agent-integration.md", + "README.md", + "docs/configuration.md", + "llms.txt" + ] + }, + { + "id": "team-5", + "query": "Why must actively written .ragmir storage stay local to each workstation?", + "expectedPaths": [ + "docs/agent-integration.md", + "README.md", + "docs/configuration.md", + "llms.txt" + ] + }, + { + "id": "cli-1", + "query": "Which rgr commands cover setup preview ingest search audit and maintenance?", + "expectedPaths": [ + "docs/cli-reference.md", + "README.md", + "packages/ragmir-core/README.md", + "packages/ragmir-core/skills/ragmir/SKILL.md" + ] + }, + { + "id": "cli-2", + "query": "How can a user explain lexical and vector contributions to a search ranking?", + "expectedPaths": [ + "docs/cli-reference.md", + "README.md", + "packages/ragmir-core/README.md", + "packages/ragmir-core/skills/ragmir/SKILL.md" + ] + }, + { + "id": "cli-3", + "query": "What bounds apply to top-k and context radius in CLI retrieval?", + "expectedPaths": [ + "docs/cli-reference.md", + "README.md", + "packages/ragmir-core/README.md", + "packages/ragmir-core/skills/ragmir/SKILL.md" + ] + }, + { + "id": "cli-4", + "query": "How does research report timeout code scan and evidence budgets?", + "expectedPaths": [ + "docs/cli-reference.md", + "README.md", + "packages/ragmir-core/README.md", + "packages/ragmir-core/skills/ragmir/SKILL.md" + ] + }, + { + "id": "cli-5", + "query": "Which command audits unsupported files without writing the index?", + "expectedPaths": [ + "docs/cli-reference.md", + "README.md", + "packages/ragmir-core/README.md", + "packages/ragmir-core/skills/ragmir/SKILL.md" + ] + }, + { + "id": "api-1", + "query": "How should a long-running Node process reuse and close a Ragmir client?", + "expectedPaths": [ + "docs/api-reference.md", + "README.md", + "packages/ragmir-core/README.md", + "packages/ragmir-core/src/client.ts", + "packages/ragmir-core/src/query.ts" + ] + }, + { + "id": "api-2", + "query": "Which TypeScript API returns cited retrieval context without synthesized prose?", + "expectedPaths": [ + "docs/api-reference.md", + "README.md", + "packages/ragmir-core/README.md", + "packages/ragmir-core/src/client.ts", + "packages/ragmir-core/src/query.ts" + ] + }, + { + "id": "api-3", + "query": "How can application code cancel a Ragmir operation or set a timeout?", + "expectedPaths": [ + "docs/api-reference.md", + "README.md", + "packages/ragmir-core/README.md", + "packages/ragmir-core/src/client.ts", + "packages/ragmir-core/src/query.ts" + ] + }, + { + "id": "api-4", + "query": "What lifecycle resources are owned by createRagmirClient?", + "expectedPaths": [ + "docs/api-reference.md", + "README.md", + "packages/ragmir-core/README.md", + "packages/ragmir-core/src/client.ts", + "packages/ragmir-core/src/query.ts" + ] + }, + { + "id": "api-5", + "query": "Which typed API expands a citation into its source context?", + "expectedPaths": [ + "docs/api-reference.md", + "README.md", + "packages/ragmir-core/README.md", + "packages/ragmir-core/src/client.ts", + "packages/ragmir-core/src/query.ts" + ] + }, + { + "id": "configuration-1", + "query": "How are stable source globs configured for a Ragmir knowledge base?", + "expectedPaths": [ + "docs/configuration.md", + "README.md", + "docs/agent-integration.md", + "packages/ragmir-core/src/config.ts", + "packages/ragmir-core/src/setup-prompt.ts" + ] + }, + { + "id": "configuration-2", + "query": "Which configuration fields select local-hash or Transformers embeddings?", + "expectedPaths": [ + "docs/configuration.md", + "README.md", + "docs/agent-integration.md", + "packages/ragmir-core/src/config.ts", + "packages/ragmir-core/src/setup-prompt.ts" + ] + }, + { + "id": "configuration-3", + "query": "How should private external source folders be authorized?", + "expectedPaths": [ + "docs/configuration.md", + "README.md", + "docs/agent-integration.md", + "packages/ragmir-core/src/config.ts", + "packages/ragmir-core/src/setup-prompt.ts" + ] + }, + { + "id": "configuration-4", + "query": "What ingestion windows independently bound files bytes chunks vectors and concurrency?", + "expectedPaths": [ + "docs/configuration.md", + "README.md", + "docs/agent-integration.md", + "packages/ragmir-core/src/config.ts", + "packages/ragmir-core/src/setup-prompt.ts" + ] + }, + { + "id": "configuration-5", + "query": "How does the nearest configured ancestor behave in a monorepo?", + "expectedPaths": [ + "docs/configuration.md", + "README.md", + "docs/agent-integration.md", + "packages/ragmir-core/src/config.ts", + "packages/ragmir-core/src/setup-prompt.ts" + ] + }, + { + "id": "upgrade-1", + "query": "How should an old Ragmir installation check compatibility after a package update?", + "expectedPaths": [ + "docs/cli-reference.md", + "README.md", + "packages/ragmir-core/README.md", + "packages/ragmir-core/src/upgrade.ts", + "packages/ragmir-core/src/upgrade.test.ts", + "CHANGELOG.md", + "llms.txt" + ] + }, + { + "id": "upgrade-2", + "query": "When does an upgrade require a staged rebuild instead of index reuse?", + "expectedPaths": [ + "docs/cli-reference.md", + "README.md", + "packages/ragmir-core/README.md", + "packages/ragmir-core/src/upgrade.ts", + "packages/ragmir-core/src/upgrade.test.ts", + "CHANGELOG.md", + "llms.txt" + ] + }, + { + "id": "upgrade-3", + "query": "Why does Ragmir never delete the active index before rebuilding?", + "expectedPaths": [ + "docs/cli-reference.md", + "README.md", + "packages/ragmir-core/README.md", + "packages/ragmir-core/src/upgrade.ts", + "packages/ragmir-core/src/upgrade.test.ts", + "CHANGELOG.md", + "llms.txt" + ] + }, + { + "id": "upgrade-4", + "query": "How can a long-running service continue serving during an index upgrade?", + "expectedPaths": [ + "docs/cli-reference.md", + "README.md", + "packages/ragmir-core/README.md", + "packages/ragmir-core/src/upgrade.ts", + "packages/ragmir-core/src/upgrade.test.ts", + "CHANGELOG.md", + "llms.txt" + ] + }, + { + "id": "upgrade-5", + "query": "Which safe defaults are applied to older configuration files?", + "expectedPaths": [ + "docs/cli-reference.md", + "README.md", + "packages/ragmir-core/README.md", + "packages/ragmir-core/src/upgrade.ts", + "packages/ragmir-core/src/upgrade.test.ts", + "CHANGELOG.md", + "llms.txt" + ] + }, + { + "id": "chat-1", + "query": "How is local GGUF answer synthesis kept separate from Ragmir Core retrieval?", + "expectedPaths": [ + "README.md", + "llms.txt" + ] + }, + { + "id": "chat-2", + "query": "What are the download sizes for lite fast and quality Chat profiles?", + "expectedPaths": [ + "README.md", + "llms.txt" + ] + }, + { + "id": "chat-3", + "query": "How does Ragmir Chat validate visible citation markers in a local answer?", + "expectedPaths": [ + "README.md", + "llms.txt" + ] + }, + { + "id": "chat-4", + "query": "Which command verifies a Chat model profile is ready for offline generation?", + "expectedPaths": [ + "README.md", + "llms.txt" + ] + }, + { + "id": "chat-5", + "query": "Does installing Ragmir Core require Qwen Gemma or the Chat package?", + "expectedPaths": [ + "README.md", + "llms.txt" + ] + }, + { + "id": "tts-1", + "query": "Which languages have an offline Ragmir TTS model?", + "expectedPaths": [ + "README.md", + "llms.txt" + ] + }, + { + "id": "tts-2", + "query": "When may narration text be sent to the Edge speech service?", + "expectedPaths": [ + "README.md", + "llms.txt" + ] + }, + { + "id": "tts-3", + "query": "How should a TTS model be preloaded with non-sensitive text?", + "expectedPaths": [ + "README.md", + "llms.txt" + ] + }, + { + "id": "tts-4", + "query": "Which output format is produced by offline Transformers speech?", + "expectedPaths": [ + "README.md", + "llms.txt" + ] + }, + { + "id": "tts-5", + "query": "Does Ragmir TTS retrieve evidence or only render caller-provided text?", + "expectedPaths": [ + "README.md", + "llms.txt" + ] + }, + { + "id": "troubleshooting-1", + "query": "What should a user inspect when Ragmir search results are weak?", + "expectedPaths": [ + "docs/troubleshooting.md", + "README.md", + "packages/ragmir-core/README.md", + "docs/cli-reference.md" + ] + }, + { + "id": "troubleshooting-2", + "query": "How can unsupported or oversized files be diagnosed?", + "expectedPaths": [ + "docs/troubleshooting.md", + "README.md", + "packages/ragmir-core/README.md", + "docs/cli-reference.md" + ] + }, + { + "id": "troubleshooting-3", + "query": "What repairs should doctor fix perform without deleting healthy storage?", + "expectedPaths": [ + "docs/troubleshooting.md", + "README.md", + "packages/ragmir-core/README.md", + "docs/cli-reference.md" + ] + }, + { + "id": "troubleshooting-4", + "query": "How should a missing optional Chat or TTS peer package be diagnosed?", + "expectedPaths": [ + "docs/troubleshooting.md", + "README.md", + "packages/ragmir-core/README.md", + "docs/cli-reference.md" + ] + }, + { + "id": "troubleshooting-5", + "query": "Which checks distinguish privacy warnings from operational readiness?", + "expectedPaths": [ + "docs/troubleshooting.md", + "README.md", + "packages/ragmir-core/README.md", + "docs/cli-reference.md" + ] + }, + { + "id": "security-1", + "query": "Which privacy controls harden a Ragmir installation for confidential documents?", + "expectedPaths": [ + "SECURITY-HARDENING.md", + "SECURITY.md", + "README.md", + "packages/ragmir-core/README.md", + "docs/configuration.md", + "docs/agent-integration.md" + ] + }, + { + "id": "security-2", + "query": "How does security-audit verify Git ignore coverage and local permissions?", + "expectedPaths": [ + "SECURITY-HARDENING.md", + "SECURITY.md", + "README.md", + "packages/ragmir-core/README.md", + "docs/configuration.md", + "docs/agent-integration.md" + ] + }, + { + "id": "security-3", + "query": "What content must never be committed from the .ragmir directory?", + "expectedPaths": [ + "SECURITY-HARDENING.md", + "SECURITY.md", + "README.md", + "packages/ragmir-core/README.md", + "docs/configuration.md", + "docs/agent-integration.md" + ] + }, + { + "id": "security-4", + "query": "Why must vulnerabilities leaked secrets and private document disclosures be reported privately instead of in a public issue?", + "expectedPaths": [ + "SECURITY-HARDENING.md", + "SECURITY.md", + "README.md", + "packages/ragmir-core/README.md", + "docs/configuration.md", + "docs/agent-integration.md" + ] + }, + { + "id": "security-5", + "query": "Why are OCR and external extractors explicit local authority boundaries?", + "expectedPaths": [ + "SECURITY-HARDENING.md", + "SECURITY.md", + "README.md", + "packages/ragmir-core/README.md", + "docs/configuration.md", + "docs/agent-integration.md" + ] + }, + { + "id": "release-1", + "query": "What validation gate must pass before a Ragmir release?", + "expectedPaths": [ + "RELEASING.md", + "README.md", + "package.json", + "AGENTS.md", + "CLAUDE.md", + "scripts/semantic-release-smoke.mjs" + ] + }, + { + "id": "release-2", + "query": "How are Core Chat and TTS package compatibility checked before publishing?", + "expectedPaths": [ + "RELEASING.md", + "README.md", + "package.json", + "AGENTS.md", + "CLAUDE.md", + "scripts/semantic-release-smoke.mjs" + ] + }, + { + "id": "release-3", + "query": "Which release notes sections describe highlights details and verification?", + "expectedPaths": [ + "RELEASING.md", + "README.md", + "package.json", + "AGENTS.md", + "CLAUDE.md", + "scripts/semantic-release-smoke.mjs" + ] + }, + { + "id": "release-4", + "query": "How is semantic-release used to publish Ragmir packages?", + "expectedPaths": [ + "RELEASING.md", + "README.md", + "package.json", + "AGENTS.md", + "CLAUDE.md", + "scripts/semantic-release-smoke.mjs" + ] + }, + { + "id": "release-5", + "query": "What should be verified after npm packages and the landing are released?", + "expectedPaths": [ + "RELEASING.md", + "README.md", + "package.json", + "AGENTS.md", + "CLAUDE.md", + "scripts/semantic-release-smoke.mjs" + ] + }, + { + "id": "contributing-1", + "query": "Which Node version and package manager should contributors use?", + "expectedPaths": [ + "CONTRIBUTING.md", + "AGENTS.md", + "CLAUDE.md", + "README.md" + ] + }, + { + "id": "contributing-2", + "query": "How should a contributor run the repository validation suite?", + "expectedPaths": [ + "CONTRIBUTING.md", + "AGENTS.md", + "CLAUDE.md", + "README.md" + ] + }, + { + "id": "contributing-3", + "query": "What code and documentation expectations apply to a Ragmir change?", + "expectedPaths": [ + "CONTRIBUTING.md", + "AGENTS.md", + "CLAUDE.md", + "README.md" + ] + }, + { + "id": "contributing-4", + "query": "How should commits be formatted in the Ragmir repository?", + "expectedPaths": [ + "CONTRIBUTING.md", + "AGENTS.md", + "CLAUDE.md", + "README.md" + ] + }, + { + "id": "contributing-5", + "query": "Which local generated files must contributors keep out of Git?", + "expectedPaths": [ + "CONTRIBUTING.md", + "AGENTS.md", + "CLAUDE.md", + "README.md" + ] + }, + { + "id": "scripts-1", + "query": "Which root script runs the complete Ragmir validation gate?", + "expectedPaths": [ + "package.json", + "README.md", + "AGENTS.md", + "CLAUDE.md", + "RELEASING.md", + "scripts/public-surface-smoke.mjs", + "scripts/semantic-release-smoke.mjs" + ] + }, + { + "id": "scripts-2", + "query": "Which package scripts build Core Chat TTS and the landing?", + "expectedPaths": [ + "package.json", + "README.md", + "AGENTS.md", + "CLAUDE.md", + "RELEASING.md", + "scripts/public-surface-smoke.mjs", + "scripts/semantic-release-smoke.mjs" + ] + }, + { + "id": "scripts-3", + "query": "What script checks the public repository surface for secrets and generated state?", + "expectedPaths": [ + "package.json", + "README.md", + "AGENTS.md", + "CLAUDE.md", + "RELEASING.md", + "scripts/public-surface-smoke.mjs", + "scripts/semantic-release-smoke.mjs" + ] + }, + { + "id": "scripts-4", + "query": "Which pnpm command runs the landing development server on localhost 4322?", + "expectedPaths": [ + "package.json", + "README.md", + "AGENTS.md", + "CLAUDE.md", + "RELEASING.md", + "scripts/public-surface-smoke.mjs", + "scripts/semantic-release-smoke.mjs", + "packages/ragmir-landing/README.md" + ] + }, + { + "id": "scripts-5", + "query": "Which release script verifies semantic-release configuration?", + "expectedPaths": [ + "package.json", + "README.md", + "AGENTS.md", + "CLAUDE.md", + "RELEASING.md", + "scripts/public-surface-smoke.mjs", + "scripts/semantic-release-smoke.mjs" + ] + }, + { + "id": "mcp-1", + "query": "Which read-focused tools are exposed by the Ragmir MCP server?", + "expectedPaths": [ + "packages/ragmir-core/src/mcp.ts", + "docs/agent-integration.md", + "docs/api-reference.md", + "README.md", + "llms.txt", + "context7.json", + "packages/ragmir-core/skills/ragmir/SKILL.md" + ] + }, + { + "id": "mcp-2", + "query": "How are MCP JSON responses bounded when the output budget is small?", + "expectedPaths": [ + "packages/ragmir-core/src/mcp.ts", + "docs/agent-integration.md", + "docs/api-reference.md", + "README.md", + "llms.txt", + "context7.json", + "packages/ragmir-core/skills/ragmir/SKILL.md" + ] + }, + { + "id": "mcp-3", + "query": "When does the MCP server refresh its lazy Ragmir client?", + "expectedPaths": [ + "packages/ragmir-core/src/mcp.ts", + "docs/agent-integration.md", + "docs/api-reference.md", + "README.md", + "llms.txt", + "context7.json", + "packages/ragmir-core/skills/ragmir/SKILL.md" + ] + }, + { + "id": "mcp-4", + "query": "How does MCP advertise destructive and open-world tool behavior?", + "expectedPaths": [ + "packages/ragmir-core/src/mcp.ts", + "docs/agent-integration.md", + "docs/api-reference.md", + "README.md", + "llms.txt", + "context7.json", + "packages/ragmir-core/skills/ragmir/SKILL.md" + ] + }, + { + "id": "mcp-5", + "query": "Which MCP resources orient an agent before it searches?", + "expectedPaths": [ + "packages/ragmir-core/src/mcp.ts", + "docs/agent-integration.md", + "docs/api-reference.md", + "README.md", + "llms.txt", + "context7.json", + "packages/ragmir-core/skills/ragmir/SKILL.md" + ] + }, + { + "id": "ingestion-1", + "query": "How does Ragmir ingestion discover redact chunk embed and commit selected files?", + "expectedPaths": [ + "packages/ragmir-core/src/ingest.ts", + "docs/cli-reference.md", + "README.md", + "packages/ragmir-core/README.md", + "docs/configuration.md", + "AGENTS.md", + "llms.txt" + ] + }, + { + "id": "ingestion-2", + "query": "How are concurrent local index writers serialized?", + "expectedPaths": [ + "packages/ragmir-core/src/ingest.ts", + "docs/cli-reference.md", + "README.md", + "packages/ragmir-core/README.md", + "docs/configuration.md", + "AGENTS.md", + "llms.txt" + ] + }, + { + "id": "ingestion-3", + "query": "What durable progress allows an interrupted ingestion to resume?", + "expectedPaths": [ + "packages/ragmir-core/src/ingest.ts", + "docs/cli-reference.md", + "README.md", + "packages/ragmir-core/README.md", + "docs/configuration.md", + "AGENTS.md", + "llms.txt" + ] + }, + { + "id": "ingestion-4", + "query": "How does a rebuild validate and activate a new generation atomically?", + "expectedPaths": [ + "packages/ragmir-core/src/ingest.ts", + "docs/cli-reference.md", + "README.md", + "packages/ragmir-core/README.md", + "docs/configuration.md", + "AGENTS.md", + "llms.txt" + ] + }, + { + "id": "ingestion-5", + "query": "When can an unchanged file be reused instead of parsed again?", + "expectedPaths": [ + "packages/ragmir-core/src/ingest.ts", + "docs/cli-reference.md", + "README.md", + "packages/ragmir-core/README.md", + "docs/configuration.md", + "AGENTS.md", + "llms.txt" + ] + }, + { + "id": "evaluation-1", + "query": "Which metrics are calculated by the Ragmir golden-query evaluator?", + "expectedPaths": [ + "packages/ragmir-core/src/evaluate.ts", + "docs/cli-reference.md", + "docs/api-reference.md", + "README.md", + "packages/ragmir-core/benchmarks/README.md" + ] + }, + { + "id": "evaluation-2", + "query": "What makes a retrieval quality report eligible for verification?", + "expectedPaths": [ + "packages/ragmir-core/src/evaluate.ts", + "docs/cli-reference.md", + "docs/api-reference.md", + "README.md", + "packages/ragmir-core/benchmarks/README.md" + ] + }, + { + "id": "evaluation-3", + "query": "How are exact citations graded in a golden query?", + "expectedPaths": [ + "packages/ragmir-core/src/evaluate.ts", + "docs/cli-reference.md", + "docs/api-reference.md", + "README.md", + "packages/ragmir-core/benchmarks/README.md" + ] + }, + { + "id": "evaluation-4", + "query": "How does a hard-negative case measure false positives?", + "expectedPaths": [ + "packages/ragmir-core/src/evaluate.ts", + "docs/cli-reference.md", + "docs/api-reference.md", + "README.md", + "packages/ragmir-core/benchmarks/README.md" + ] + }, + { + "id": "evaluation-5", + "query": "What invalidates a stored retrieval quality fingerprint?", + "expectedPaths": [ + "packages/ragmir-core/src/evaluate.ts", + "docs/cli-reference.md", + "docs/api-reference.md", + "README.md", + "packages/ragmir-core/benchmarks/README.md" + ] + }, + { + "id": "landing-1", + "query": "Which structured data entities describe the Ragmir landing page?", + "expectedPaths": [ + "packages/ragmir-landing/src/pages/[...locale]/index.astro", + "packages/ragmir-landing/src/components/library-section.tsx", + "packages/ragmir-landing/messages/en.json", + "packages/ragmir-landing/messages/fr.json", + "packages/ragmir-landing/README.md", + "packages/ragmir-landing/src/public-contract.test.ts", + "scripts/public-surface-smoke.mjs" + ] + }, + { + "id": "landing-2", + "query": "How does the landing keep English and French metadata aligned?", + "expectedPaths": [ + "packages/ragmir-landing/src/pages/[...locale]/index.astro", + "packages/ragmir-landing/src/components/library-section.tsx", + "packages/ragmir-landing/messages/en.json", + "packages/ragmir-landing/messages/fr.json", + "packages/ragmir-landing/README.md", + "packages/ragmir-landing/src/public-contract.test.ts", + "scripts/public-surface-smoke.mjs" + ] + }, + { + "id": "landing-3", + "query": "Where does the landing present the copyable guided setup prompt?", + "expectedPaths": [ + "packages/ragmir-landing/src/pages/[...locale]/index.astro", + "packages/ragmir-landing/src/components/library-section.tsx", + "packages/ragmir-landing/messages/en.json", + "packages/ragmir-landing/messages/fr.json", + "packages/ragmir-landing/README.md", + "packages/ragmir-landing/src/public-contract.test.ts", + "scripts/public-surface-smoke.mjs" + ] + }, + { + "id": "landing-4", + "query": "Which package-manager tabs are available for manual installation?", + "expectedPaths": [ + "packages/ragmir-landing/src/pages/[...locale]/index.astro", + "packages/ragmir-landing/src/components/library-section.tsx", + "packages/ragmir-landing/messages/en.json", + "packages/ragmir-landing/messages/fr.json", + "packages/ragmir-landing/README.md", + "packages/ragmir-landing/src/public-contract.test.ts", + "scripts/public-surface-smoke.mjs" + ] + } + ], + "unrelated": [ + "What is the weather forecast for Lyon tomorrow?", + "Quel temps fera-t-il à Bordeaux ce week-end ?", + "How do I make a classic French onion soup?", + "Recette de la tarte tatin aux pommes", + "Who won the 2018 FIFA World Cup final?", + "Qui a remporté le Tour de France 2019 ?", + "What is the capital of Australia?", + "Combien de lunes possède la planète Jupiter ?", + "How tall is the Eiffel Tower?", + "Quelle est la meilleure période pour visiter le Japon ?", + "What is the boiling point of water at the top of Mount Everest?", + "Comment faire pousser des tomates sur un balcon ?", + "Who painted the Mona Lisa?", + "Quel est le prix moyen d'un billet de train Paris Marseille ?", + "How do I change a flat tire on a bicycle?", + "Quelle race de chien est la plus calme pour un appartement ?", + "What are the rules of cricket?", + "Comment détartrer une bouilloire avec du vinaigre ?", + "When did the Roman Empire fall?", + "Quels sont les symptômes d'un rhume chez le chat ?", + "Best hiking trails near Chamonix", + "Quelle est la hauteur du Mont Blanc ?", + "How does a sourdough starter work?", + "qwzx plorbnik vantrel ?", + "blorf glimmer tasky snarl", + "zxcv asdf qwer uiop", + "Lorem ipsum dolor sit amet consectetur", + "Où trouver des champignons en forêt en octobre ?", + "What year did the Berlin Wall fall?", + "Quel vin servir avec une raclette ?" + ], + "nearDomain": [ + "How does Ragmir expose a GraphQL API?", + "Where is the Helm chart to deploy Ragmir on Kubernetes?", + "How does Ragmir sync with a Pinecone index?", + "How do I configure the Redis cache for Ragmir search results?", + "Which command exports the Ragmir index to Parquet?", + "Is there a Windows MSI installer for Ragmir?", + "How do I connect Ragmir to a Slack bot?", + "How does the Ragmir Discord integration work?", + "How do I index Confluence spaces with Ragmir?", + "How does the Ragmir SharePoint connector authenticate?", + "How do I sync a Notion workspace into Ragmir?", + "How does Ragmir encrypt the index with AWS KMS?", + "How do I configure SAML single sign-on for Ragmir?", + "Which Prometheus metrics does Ragmir export?", + "Where is the Ragmir Java SDK documented?", + "How do I stream Ragmir search results over server-sent events?", + "How do I index AutoCAD DWG drawings with Ragmir?", + "How do I index Outlook PST email archives?", + "How do I configure the built-in web crawler to index a website?", + "How do I index YouTube video transcripts with Ragmir?", + "Which Elasticsearch version does Ragmir require?", + "How do I configure Ragmir with Azure OpenAI embeddings?", + "How does Ragmir index WhatsApp or iMessage chats?", + "How does Ragmir handle Git LFS files during ingestion?", + "How do I index a MongoDB collection with Ragmir?", + "How does the GraphRAG knowledge graph builder in Ragmir work?", + "How do I configure the Ragmir Chrome extension?", + "How does Ragmir OCR Arabic right-to-left scans?", + "How does Ragmir handle Excel macros in .xlsm files?", + "How does Ragmir federated search query several remote servers?" + ] +} diff --git a/packages/ragmir-core/package.json b/packages/ragmir-core/package.json index 46b37f1..7b6f3e2 100644 --- a/packages/ragmir-core/package.json +++ b/packages/ragmir-core/package.json @@ -63,6 +63,7 @@ }, "scripts": { "bench:compare": "node benchmarks/compare.mjs", + "bench:competitors": "pnpm build && node benchmarks/competitors.mjs", "bench:cli-startup": "pnpm build && node benchmarks/cli-startup.mjs", "bench:concurrency": "pnpm build && node benchmarks/concurrency.mjs", "bench:discovery": "pnpm build && node benchmarks/discovery.mjs", @@ -118,10 +119,14 @@ }, "devDependencies": { "@huggingface/transformers": "^4.3.1", + "@langchain/core": "^1.3.2", + "@langchain/textsplitters": "^1.0.2", + "@orama/orama": "^3.1.18", "@types/html-to-text": "^9.0.4", "@types/node": "^22.20.5", "@types/picomatch": "^4.0.3", "apache-arrow": "^18.1.0", + "minisearch": "^7.2.0", "publint": "^0.3.25", "typescript": "^7.0.2", "vitest": "^5.0.5" diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 818a31f..0f0e246 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -30,9 +30,15 @@ importers: '@vitest/coverage-v8': specifier: 5.0.5 version: 5.0.5(vitest@5.0.5) + knip: + specifier: ^6.44.1 + version: 6.44.1 semantic-release: specifier: ^25.0.9 version: 25.0.9(supports-color@7.2.0)(typescript@7.0.2) + typescript: + specifier: ^7.0.2 + version: 7.0.2 yaml: specifier: ^2.9.1 version: 2.9.1 @@ -44,7 +50,7 @@ importers: version: 0.30.0(apache-arrow@18.1.0) '@modelcontextprotocol/sdk': specifier: ^1.32.2 - version: 1.32.2(supports-color@7.2.0)(zod@4.6.7) + version: 1.32.2(@cfworker/json-schema@4.1.4)(supports-color@7.2.0)(zod@4.6.7) commander: specifier: ^15.0.0 version: 15.0.0 @@ -82,6 +88,15 @@ importers: '@huggingface/transformers': specifier: ^4.3.1 version: 4.3.1(@types/node@22.20.5) + '@langchain/core': + specifier: ^1.3.2 + version: 1.3.2(@opentelemetry/api@1.10.0) + '@langchain/textsplitters': + specifier: ^1.0.2 + version: 1.0.2(@langchain/core@1.3.2(@opentelemetry/api@1.10.0)) + '@orama/orama': + specifier: ^3.1.18 + version: 3.1.18 '@types/html-to-text': specifier: ^9.0.4 version: 9.0.4 @@ -94,6 +109,9 @@ importers: apache-arrow: specifier: ^18.1.0 version: 18.1.0 + minisearch: + specifier: ^7.2.0 + version: 7.2.0 publint: specifier: ^0.3.25 version: 0.3.25 @@ -461,6 +479,9 @@ packages: resolution: {integrity: sha512-CuNiSqg7+e1cO/GjffyMOm5Tt2jUF9CWHHnvQ/UkqvtkGfHdgwEC0wpmq7fkN3gxwpRnrAN0WzO3vREKmNolMQ==} engines: {node: '>=18'} + '@cfworker/json-schema@4.1.4': + resolution: {integrity: sha512-20rPE8EX0dSg6JGOVe31HpqsLYZTDSlcFJZA5mpZqQxcZ71vh/SXCMX+UuHO1mMcZBDKeb752z+3L8IWa+0Svg==} + '@clack/core@1.5.1': resolution: {integrity: sha512-iHTrHA8MtVuLl2TfZySmcKv1qO2PoyC9Z7pfSDozEuV5vtY3/wcOPKJXlqJ5Oq2Cx5DDGQGAMVx6HZfRRoVEbQ==} engines: {node: '>= 20.12.0'} @@ -582,12 +603,18 @@ packages: '@emnapi/core@1.11.1': resolution: {integrity: sha512-RSvbQmHzdKzNsLYa/wHrbc3KN4sYLKAdPZxqiM2HATqv/SBk2/ENSHpvXGaLOMcsAyz0poEGqkmmKYG3OWiJEQ==} + '@emnapi/core@1.11.2': + resolution: {integrity: sha512-TC8MkTuZUtcTSiFeuC0ksCh9QIJ5+F21MvZ4Wn4ORfYaFJ/0dsiudv5tVkejgwZlwQ39jL9WWDe2lz8x0WglOA==} + '@emnapi/core@1.11.3': resolution: {integrity: sha512-zLpS5asjEb7lq8jYLq37N6XKaE41DIexlY1rF/z4/tIl3wo13Sqm28fRyfIsKZD+NZ8mM5RoKkpW/rBcuoSZSg==} '@emnapi/runtime@1.11.1': resolution: {integrity: sha512-vgj7R3y3Wgx24IQaGPA/R6YFXLHVMOZ0uVEyIQPaWs+rd1AzfEMXlAC22FYwO1XkKR6NPsq7mUandH8oIRdZFw==} + '@emnapi/runtime@1.11.2': + resolution: {integrity: sha512-kyOl3X0DuTiT1h2ft8r2fYO8JYtU9a9Xis/zBSiGArNaagCOWx90N1k2wxp18czFDH+OgcWGb5ZP/XMt3dcyPA==} + '@emnapi/runtime@1.11.3': resolution: {integrity: sha512-Xz4Tpyki7XyrpbUK1jR1AhdAdaXyhhY4lZ3neLodmhpuWfy2PAQN5B46sAiU4liOXGLkHypn/qU+jvfWSCYYLA==} @@ -1001,6 +1028,16 @@ packages: peerDependencies: apache-arrow: '>=15.0.0 <=18.1.0' + '@langchain/core@1.3.2': + resolution: {integrity: sha512-kXI5wkIRy6MVNBFzFdN4MyNAGPTsodVZFQ9BRQTyOCNnKy+qPuipMCl+qaybtN6rQ+0YrJvfTxYd+u4goAHtFQ==} + engines: {node: '>=20'} + + '@langchain/textsplitters@1.0.2': + resolution: {integrity: sha512-UNCmlyFIRlRfVxqpmcRsp1jQrscnIVVqRKrrjFF2wHHq+VGTv+VDkGZLyEVUopJfHIdaiXfOKlSrROQui7VLFA==} + engines: {node: '>=20'} + peerDependencies: + '@langchain/core': ^1.0.0 + '@lucide/astro@1.61.0': resolution: {integrity: sha512-XoFZGoHjYtnDTuOQ4b105IO+WUCh8rzitIgBIrkOcdC1PRmsvLx+fwnrCRHiZ1JfbA36r/jAMxQL/HeyRqKJjQ==} peerDependencies: @@ -1100,12 +1137,241 @@ packages: resolution: {integrity: sha512-lMkyes8iHrG+mk6WH7X70MdCLI2RS0FwNBY1PyQ8UemaijE2m5mv5A0Niy3HtbYTFrgkUEQjFMXmSAk2hQDuPQ==} engines: {node: '>=8.0.0'} + '@orama/orama@3.1.18': + resolution: {integrity: sha512-a61ljmRVVyG5MC/698C8/FfFDw5a8LOIvyOLW5fztgUXqUpc1jOfQzOitSCbge657OgXXThmY3Tk8fpiDb4UcA==} + engines: {node: '>= 20.0.0'} + '@oslojs/encoding@1.1.0': resolution: {integrity: sha512-70wQhgYmndg4GCPxPPxPGevRKqTIJ2Nh4OkiMWmDAVYsTQ+Ta7Sq+rPevXyXGdzr30/qZBnyOalCszoMxlyldQ==} + '@oxc-parser/binding-android-arm-eabi@0.155.0': + resolution: {integrity: sha512-A6xv7hDtnTm+xlZ7utn6rRQpRMJHRhVr3gjRGqlx0RUX84mNk0pAsy6YN/wWhuOTkaZhRrrzaLxf0NfbH77j2w==} + engines: {node: ^20.19.0 || >=22.12.0} + cpu: [arm] + os: [android] + + '@oxc-parser/binding-android-arm64@0.155.0': + resolution: {integrity: sha512-T1wWwH9/JI5g7Z4kEQwDOwhToxdDhRyOTklH+uuNpJ16m3MZFmGl+l2rKc0NHQdbVRG2LTOBsNjpW3h1BWytPA==} + engines: {node: ^20.19.0 || >=22.12.0} + cpu: [arm64] + os: [android] + + '@oxc-parser/binding-darwin-arm64@0.155.0': + resolution: {integrity: sha512-ZoRB3qJxJ8DWA41jHja+YeB0lLfjBy4T+Kr15vnN1k65X3zbJt3rPZftEwN4Ju4tmA4K/T9dWWD7Z8ewQLnrVA==} + engines: {node: ^20.19.0 || >=22.12.0} + cpu: [arm64] + os: [darwin] + + '@oxc-parser/binding-darwin-x64@0.155.0': + resolution: {integrity: sha512-vxtL86BtM+IgrpejA86nbTk5j0HKTIMeuaJ1fQciw6p9RfavPtIPjI7gN34owap0yG9txXBaAWpGdxokNwdqIw==} + engines: {node: ^20.19.0 || >=22.12.0} + cpu: [x64] + os: [darwin] + + '@oxc-parser/binding-freebsd-x64@0.155.0': + resolution: {integrity: sha512-tN6Ru5e0UVnWaIuiH8jEwnvxuxg7FvhWwSd3oe5oDhOD4ObsSr9/N6WfHLjilUWlPPj65ptwrSDH0Y7lzwTzmw==} + engines: {node: ^20.19.0 || >=22.12.0} + cpu: [x64] + os: [freebsd] + + '@oxc-parser/binding-linux-arm-gnueabihf@0.155.0': + resolution: {integrity: sha512-h4ozA5n6L6lwolsz9iwunEhKWnIVbmFRots5QxEix0H6nJ5ICpuDa7PuShuhfpIDFe3+Qvyscioyiv+9g+4Syg==} + engines: {node: ^20.19.0 || >=22.12.0} + cpu: [arm] + os: [linux] + + '@oxc-parser/binding-linux-arm-musleabihf@0.155.0': + resolution: {integrity: sha512-058/+LardpfuUlsKIvBGk/CYoakEVcMgqYx8GgyUZeOxFWTojYLlCc1Gb60/y21v3esDhHidgiY/Zbq9LpBhaQ==} + engines: {node: ^20.19.0 || >=22.12.0} + cpu: [arm] + os: [linux] + + '@oxc-parser/binding-linux-arm64-gnu@0.155.0': + resolution: {integrity: sha512-nrOt2Y3xIAzu+lSPzH0864ieO+iCCXgoo9qFlGANE8Xh0cQ+RP8Wstfp+/wXN2YCyb98cAGzX0VruKYw7it/jw==} + engines: {node: ^20.19.0 || >=22.12.0} + cpu: [arm64] + os: [linux] + libc: [glibc] + + '@oxc-parser/binding-linux-arm64-musl@0.155.0': + resolution: {integrity: sha512-KYAayyN5hQbJPtGKtRxlyc8XYmyntOplyXSQzTDrmbz7l/GCrzQbSswvyJFBTH1LGtsZrbXOAfN52i/Kz6kLKg==} + engines: {node: ^20.19.0 || >=22.12.0} + cpu: [arm64] + os: [linux] + libc: [musl] + + '@oxc-parser/binding-linux-ppc64-gnu@0.155.0': + resolution: {integrity: sha512-CSuSY5hfjtsBnHXydOj8jiNfS1qHbdPZuPz1YL1cSgK7VmNYP+QlZIbG0BBYYBgX/vYEgZwbG9mBdW4YairZ5w==} + engines: {node: ^20.19.0 || >=22.12.0} + cpu: [ppc64] + os: [linux] + libc: [glibc] + + '@oxc-parser/binding-linux-riscv64-gnu@0.155.0': + resolution: {integrity: sha512-7KCfOQm+Nz2N8trAXbQkal4TQ5756WVEmhm35y816YHvyFnEfVsICsiV5k0pEszNf/fRm4cm2i6i18SJ6/iCWg==} + engines: {node: ^20.19.0 || >=22.12.0} + cpu: [riscv64] + os: [linux] + libc: [glibc] + + '@oxc-parser/binding-linux-riscv64-musl@0.155.0': + resolution: {integrity: sha512-JSlhgoK+/g1TGJI5NuADx2+6syhxnQNLCWLXyO5QLFyetomcr/vIxSdiNlmyAv963YXUzCuwAwRG6npZ5xyJ1A==} + engines: {node: ^20.19.0 || >=22.12.0} + cpu: [riscv64] + os: [linux] + libc: [musl] + + '@oxc-parser/binding-linux-s390x-gnu@0.155.0': + resolution: {integrity: sha512-WFIX+dPokWp64mi434Md3U3lJaTwNxAphvGjpltoyQjGUsdXavOIRrnp9dOS34Hd75HOAwDxAI+YwWOwqmJbtg==} + engines: {node: ^20.19.0 || >=22.12.0} + cpu: [s390x] + os: [linux] + libc: [glibc] + + '@oxc-parser/binding-linux-x64-gnu@0.155.0': + resolution: {integrity: sha512-g6xSDC/JpbnVOz5uafMGyK3txBVRFMQTx8opv/yHXGxqV3ZMVl3bf7W8XrSnF68dmy3gM4dgReNryuE53+WkeQ==} + engines: {node: ^20.19.0 || >=22.12.0} + cpu: [x64] + os: [linux] + libc: [glibc] + + '@oxc-parser/binding-linux-x64-musl@0.155.0': + resolution: {integrity: sha512-6/4xlxKlZJWrQ0uwDFPc0H9+xLu7068CKhX3vIDW3SbvP/lVgnVuT5v6dyugeWskWpuxqF/WJ4WKwhxjU3pG2w==} + engines: {node: ^20.19.0 || >=22.12.0} + cpu: [x64] + os: [linux] + libc: [musl] + + '@oxc-parser/binding-openharmony-arm64@0.155.0': + resolution: {integrity: sha512-tltklL0J0XjDxIRTBXWOBhvc2mitcmES45pIxhwDWfxRyqz7Vkeu0NekTZEIxGE4f/lXgq3vZcjjffteQOVorg==} + engines: {node: ^20.19.0 || >=22.12.0} + cpu: [arm64] + os: [openharmony] + + '@oxc-parser/binding-win32-arm64-msvc@0.155.0': + resolution: {integrity: sha512-alAb9TzjatUEsOeoncAziUnfdTDxfyswhalFZ9sC6plUgATIUx2NIZ0+bMn6aJ28AduS1OpG/OAisr/YISn4Ig==} + engines: {node: ^20.19.0 || >=22.12.0} + cpu: [arm64] + os: [win32] + + '@oxc-parser/binding-win32-ia32-msvc@0.155.0': + resolution: {integrity: sha512-yc5cbSfcGmRv/KpxDFuyk8adEBaWd/9HilWvBbQ4bFbus1V2PUh2mGB2Cfq9rqZTqLeiMT8x0skV+LKp9hOMgQ==} + engines: {node: ^20.19.0 || >=22.12.0} + cpu: [ia32] + os: [win32] + + '@oxc-parser/binding-win32-x64-msvc@0.155.0': + resolution: {integrity: sha512-8heq6jGq+rfpTDRDjoQiBHw0maQZf3IXkS99pdHQZPoTL5kyF5qyPS0RidUlSdnWDIqxXb5nux7GBD9cT+imUw==} + engines: {node: ^20.19.0 || >=22.12.0} + cpu: [x64] + os: [win32] + '@oxc-project/types@0.155.0': resolution: {integrity: sha512-SeE8JPwW1xs1tHeNycjUtUBdhorRa8o4jlvjwk49R7/Z291i+4DQa7JJIgOm/TaYQZx+QrPQbVJ/9c8dMIZN8Q==} + '@oxc-resolver/binding-android-arm-eabi@11.24.2': + resolution: {integrity: sha512-y09e0L0SRI2OA2tUIrjBgoV3eH5hvUKXNkJqXmNo5V2WxIjyC7I7aJfRLMEVpA8yi95f90gFDvO0VMgrDw+vwA==} + cpu: [arm] + os: [android] + + '@oxc-resolver/binding-android-arm64@11.24.2': + resolution: {integrity: sha512-cl4icWaZFnLdg8m6qtnh5rBMuGbxc/ptStFHLeCNwr+2cZjkjNwQu/jYRS0CHlnPecOJMpuS5M6/BH+0J/YkEg==} + cpu: [arm64] + os: [android] + + '@oxc-resolver/binding-darwin-arm64@11.24.2': + resolution: {integrity: sha512-At29QEMF6HajbQvgY8K6OXnHD1x9rad74xBEfmCB6ZqCGsdq75aK7tOYcTbOanMy8qdIBrfL3SMr3p/lfSlb9w==} + cpu: [arm64] + os: [darwin] + + '@oxc-resolver/binding-darwin-x64@11.24.2': + resolution: {integrity: sha512-A5Kqr1EUj4oIL5CF4WRssq/o5P0Y11cwoFouMRmQ7YnC/A8V93nv1nb7aSU8HwcgmXropjLNkVTl4MN87cu28Q==} + cpu: [x64] + os: [darwin] + + '@oxc-resolver/binding-freebsd-x64@11.24.2': + resolution: {integrity: sha512-R5xkRBRRz7ceH/P5Jrc6G7FmdUdgpLYyESFAUDVTNQ9K0sGPxcp4ljiwEwEqsvNcQ4sYbMRrWcHHBCu7ksAJVw==} + cpu: [x64] + os: [freebsd] + + '@oxc-resolver/binding-linux-arm-gnueabihf@11.24.2': + resolution: {integrity: sha512-k/RuYL4L/R58IBn3wT5ma3Wh4k62bp1eYCFRWCmMsasUOqL+H6sW0VGFadEzKWXFFlz+2uIMoeMk9ySSZJHgbg==} + cpu: [arm] + os: [linux] + + '@oxc-resolver/binding-linux-arm-musleabihf@11.24.2': + resolution: {integrity: sha512-bnHAak3ujYfH5pKk4NieFNbvYvernfoQDgwLddbZ3OtMYrem87/qjlA+u+aKG0oZcqSLGCful/6/CEA+aeAgaA==} + cpu: [arm] + os: [linux] + + '@oxc-resolver/binding-linux-arm64-gnu@11.24.2': + resolution: {integrity: sha512-vDT3KHgzYp47gmtNOqL2VNhCyl5Zv643eyxm//A68J8DeUGXrvD1pZFiaT4jSfe+RInfnn1R2yVHye4enx6RnA==} + cpu: [arm64] + os: [linux] + libc: [glibc] + + '@oxc-resolver/binding-linux-arm64-musl@11.24.2': + resolution: {integrity: sha512-+kMlQvbzfyEYtu5FcjE4p+ttBLpKW4d/AsAsuE69BxV6V4twZJeIQZFfD8gh/wqglY0MkPSezWXQH0jBV13MUw==} + cpu: [arm64] + os: [linux] + libc: [musl] + + '@oxc-resolver/binding-linux-ppc64-gnu@11.24.2': + resolution: {integrity: sha512-shjfMhmZ3gq9fv/w7bi3PnZlgOPG+2QAOFf0BJF0EgBSIGZ6PMLN2zbGEblTUYB/NKVDRyYhE2ff3dJ1QqNPkA==} + cpu: [ppc64] + os: [linux] + libc: [glibc] + + '@oxc-resolver/binding-linux-riscv64-gnu@11.24.2': + resolution: {integrity: sha512-zGelwFR5oRo+b69k8Lrzun86DyUHzfKN6cnjbR9l7Z7NIRznOE/2ZvPa1IUKqAL2PzAXOdwkfVqNvO1H2RlpAw==} + cpu: [riscv64] + os: [linux] + libc: [glibc] + + '@oxc-resolver/binding-linux-riscv64-musl@11.24.2': + resolution: {integrity: sha512-qxZ1SWCXJY0eyhAlP6Lmo9F2Nrtx7EkYj9oCgL8apDPCwXwCEDA2U697bbT81JIc2IrVjxO4KX6WU2N+oN9Z4w==} + cpu: [riscv64] + os: [linux] + libc: [musl] + + '@oxc-resolver/binding-linux-s390x-gnu@11.24.2': + resolution: {integrity: sha512-sGCecF3cx2DFlH4t/z7ApnOnXqN48p5p5mlHDEnHTAukQa2P+qMVE4CwyWE9W+q/m3QJ7kKfGrIjax31f44oFQ==} + cpu: [s390x] + os: [linux] + libc: [glibc] + + '@oxc-resolver/binding-linux-x64-gnu@11.24.2': + resolution: {integrity: sha512-k/VlMMcSzMlahb3/fENM4rTlsJ0s3fFROA0KXPBmKggqmTSaE383sl8F3KCOXPLmVsYfW6hCitMhXCEtNeZxxg==} + cpu: [x64] + os: [linux] + libc: [glibc] + + '@oxc-resolver/binding-linux-x64-musl@11.24.2': + resolution: {integrity: sha512-8hbnZyNi97b/8wapYaIF9+t9GmZKBW2vunaOc3h9HGJptH7b7XpvZqOTBSm/MpTjr7H497BlgOaSfLUdhmy2bw==} + cpu: [x64] + os: [linux] + libc: [musl] + + '@oxc-resolver/binding-openharmony-arm64@11.24.2': + resolution: {integrity: sha512-MvyGik3a6pVgZ0t/kWlbmFxFLmXQJwgLsY2eYFHLpy0wGwRbfzeIGgDwQ3kXqE30z+kSXennRkCrT7TUvkptNg==} + cpu: [arm64] + os: [openharmony] + + '@oxc-resolver/binding-wasm32-wasi@11.24.2': + resolution: {integrity: sha512-vHcssMPwO08RTvj/c0iOBz90attxyG3wQJ0dTcyEQK43LRpcdLWZlV5feBhv6Isn6ahbQIzHbCgfa81+RiML0Q==} + engines: {node: '>=14.0.0'} + cpu: [wasm32] + + '@oxc-resolver/binding-win32-arm64-msvc@11.24.2': + resolution: {integrity: sha512-uokJqro2iBqkFvJdKQLP7d8/BUmFwESQFVmIJUQKj1Xn1a/LysJoe1vmeECLF5b3jsV8CAL5sEMJXX6SdK9Nhg==} + cpu: [arm64] + os: [win32] + + '@oxc-resolver/binding-win32-x64-msvc@11.24.2': + resolution: {integrity: sha512-UqGPmo56KDfLlfXFAFIrNflHT8tFxWGEivWg3Zeyp4Uy2NlKN1FGPr6/BxcLGG3+kZ6Wp14g5Uj+n71boqZfiw==} + cpu: [x64] + os: [win32] + '@pnpm/config.env-replace@1.1.0': resolution: {integrity: sha512-htyl8TWnKL7K/ESFa1oW2UB5lVDxuF5DpM7tBi6Hu2LNL3mWkIzNLG6N4zoCUP1lCKNxWy/3iu8mS8MvToGd6w==} engines: {node: '>=12.22.0'} @@ -1499,6 +1765,9 @@ packages: resolution: {integrity: sha512-tlqY9xq5ukxTUZBmoOp+m61cqwQD5pHJtFY3Mn8CA8ps6yghLH/Hw8UPdqg4OLmFW3IFlcXnQNmo/dh8HzXYIQ==} engines: {node: '>=18'} + '@standard-schema/spec@1.1.0': + resolution: {integrity: sha512-l2aFy5jALhniG5HgqrD6jXLi/rUWrKvqN/qJx6yoJsgKhblVd+iqqU4RCXavm/jPityDo5TCvKMnpjKnOriy0w==} + '@swc/helpers@0.5.23': resolution: {integrity: sha512-5lSsMOTXURePglDfvuAQUqkGek9Hg2kksOYay2m0+XR++b2NWYL/4sWyuvVBIs8oKnJaxkdi9whaL/sqN13afw==} @@ -2463,6 +2732,9 @@ packages: resolution: {integrity: sha512-aIL5Fx7mawVa300al2BnEE4iNvo1qETxLrPI/o05L7z6go7fCw1J6EQmbK4FmJ2AS7kgVF/KEZWufBfdClMcPg==} engines: {node: '>= 0.6'} + eventemitter3@4.0.7: + resolution: {integrity: sha512-8guHBZCwKnFhYdHr2ysuRWErTwhoN2X8XELRlrRwpmfeY2jjuUN4taQMsULKUVo1K4DvZl+0pgfyoysHxvmvEw==} + eventemitter3@5.0.4: resolution: {integrity: sha512-mlsTRyGaPBjPedk6Bvw+aqbsXDtoAyAzm5MO7JgU+yVRyMQ5O8bD4Kcci7BS85f93veegeCPkL8R4GLClnjLFw==} @@ -2521,6 +2793,9 @@ packages: fastq@1.20.3: resolution: {integrity: sha512-XKv5nnLs6nLF71NgiKJLIZFLkPyIEuOselLG7ujZnGrRfQK8HpvY+WqKhAJUAdLomwVHErVS4LfxFlPq0/FTAw==} + fd-package-json@2.0.0: + resolution: {integrity: sha512-jKmm9YtsNXN789RS/0mSzOC1NUq9mkVd65vbSSVsKdjGvYXBuE4oWe2QOEoFeRmJg+lPuZxpmrfFclNhoRMneQ==} + fdir@6.5.0: resolution: {integrity: sha512-tIbYtZbucOs0BRGqPJkshJUYdL+SDH7dVM8gjy+ERp3WAUjLEFJE+02kanyHtwjWOnwrKYBiwAmM0p4kLJAnXg==} engines: {node: '>=12.0.0'} @@ -2585,6 +2860,11 @@ packages: resolution: {integrity: sha512-Wp1zXWPVUPBmfoa3Cqc9ctaKuzKAV6uLstRqlR56kSjplf5uAce+qeyYym7F+PHbGTk+tCEdkCW6RD7DX/gBZw==} engines: {node: '>=20'} + formatly@0.11.4: + resolution: {integrity: sha512-TeCC10lP8KCgoX+Fe5lryILIT0/Dy+nP0OSVnsuB5CwXoLuOdqxCYlPFF1inR8S/zTy1ec7TjhS2TeRKWzzImA==} + engines: {node: '>=18.14.0'} + hasBin: true + forwarded@0.2.0: resolution: {integrity: sha512-buRG0fpBtRHSTCOASe6hD258tEubFoRLb4ZNA6NxMVHNw2gOcwHo9wyablzMzOA5z9xA9L1KNjk/Nt6MT9aYow==} engines: {node: '>= 0.6'} @@ -2637,6 +2917,9 @@ packages: resolution: {integrity: sha512-kVCxPF3vQM/N0B1PmoqVUqgHP+EeVjmZSQn+1oCRPxd2P21P2F19lIgbR3HBosbB1PUhOAoctJnfEn2GbN2eZA==} engines: {node: '>=18'} + get-tsconfig@4.14.3: + resolution: {integrity: sha512-++QEw4DIY7WGoukz+/+A/8dGYPT9l9yIadnmSgZ8Rjr3YVSVDipQSO9CdnJo9ePqFqUUqh+wk9uIaoiAwsiPkA==} + get-tsconfig@5.0.0-beta.6: resolution: {integrity: sha512-X6fBC0pmImC70gvX2zm56go9hx0MyoGVdG0tUCkg/D+Xnh5TJsOZ7iDbOdI3PvmtrDxnu1YdDufpK2QJX1Meqw==} engines: {node: '>=20.20.0'} @@ -2883,6 +3166,9 @@ packages: jose@6.2.13: resolution: {integrity: sha512-7TIg10ZE1x82YediN+lcrPNvGMKge9yFHQD+Qho/b8y2adfugU3AsvvHfxV0fzyFllT6l9xCjf4RWY9g/Z9PJQ==} + js-tiktoken@1.0.21: + resolution: {integrity: sha512-biOj/6M5qdgx5TKjDnFT1ymSpM5tbd3ylwDtrQvFQSu0Z7bBYko2dF+W/aUkXUPuk6IVpRxk/3Q2sHOzGlS36g==} + js-tokens@10.0.0: resolution: {integrity: sha512-lM/UBzQmfJRo9ABXbPWemivdCW8V2G8FHaHdypQaIy523snUjog0W71ayWXTjiR+ixeMyVHN2XcpnTd/liPg/Q==} @@ -2928,6 +3214,31 @@ packages: resolution: {integrity: sha512-o+NO+8WrRiQEE4/7nwRJhN1HWpVmJm511pBHUxPLtp0BUISzlBplORYSmTclCnJvQq2tKu/sgl3xVpkc7ZWuQQ==} engines: {node: '>=6'} + knip@6.44.1: + resolution: {integrity: sha512-Hbd8nvkIsNgFfXgC7Jsb5LHv7jj6VwA3tmfHd53XPX2z9Vc9RnAZqV0BIiP4OrtYqKwCoLseFHQZerb1bNlbSw==} + engines: {node: ^20.19.0 || >=22.12.0} + hasBin: true + + langsmith@0.11.6: + resolution: {integrity: sha512-5hfJM5DEZpXFxUS3U9nY2CMyBRmFnhkJj7znUODJ8vsr/Jx38tGJl4GIhV484xAY7yQFWrFOTDr33PB95kHUKg==} + peerDependencies: + '@opentelemetry/api': '*' + '@opentelemetry/exporter-trace-otlp-proto': '*' + '@opentelemetry/sdk-trace-base': '*' + openai: '*' + ws: '>=7' + peerDependenciesMeta: + '@opentelemetry/api': + optional: true + '@opentelemetry/exporter-trace-otlp-proto': + optional: true + '@opentelemetry/sdk-trace-base': + optional: true + openai: + optional: true + ws: + optional: true + leac@0.7.0: resolution: {integrity: sha512-qMrZeyEekgdRQ9o6a4NAB2EQZrv827GJdn1vnapwSJ90hWRB4TzUSunvacPkxQ2TnNqHNI1/zSt0hlo0crG8Jw==} @@ -3240,6 +3551,9 @@ packages: minimist@1.2.8: resolution: {integrity: sha512-2yyAR8qBkN3YuheJanUpWC5U3bb5osDywNB8RzDVlDwDHbocAJveqqj1u8+SVD7jkWT4yvsHCpWqqWqAxb0zCA==} + minisearch@7.2.0: + resolution: {integrity: sha512-dqT2XBYUOZOiC5t2HRnwADjhNS2cecp9u+TJRiJ1Qp/f5qjkeT5APcGPjHw+bz89Ms8Jp+cG4AlE+QZ/QnDglg==} + mri@1.2.0: resolution: {integrity: sha512-tzzskb3bG8LvYGFF/mDTpq3jpI6Q9wc3LEmBaghu+DdCssd1FakN7Bc0hVNmEyGq1bq3RgfkCb3cmQLpNPOroA==} engines: {node: '>=4'} @@ -3254,6 +3568,10 @@ packages: muggle-string@0.4.1: resolution: {integrity: sha512-VNTrAak/KhO2i8dqqnqnAHOa3cYBwXEZe9h+D5h/1ZqFSTEFHdM65lR7RoIqq3tBBYavsOXV84NoHXZ0AkPyqQ==} + mustache@4.2.0: + resolution: {integrity: sha512-71ippSywq5Yb7/tVYyGbkBggbU8H3u5Rz56fH60jGFgr8uHwxs+aSKeqmluIVzM0m0kB7xQjKS6qPfd0b2ZoqQ==} + hasBin: true + mz@2.7.0: resolution: {integrity: sha512-z81GNO7nnYMEhrGh9LeymoE4+Yr0Wn5McHIZMK5cfQCl+NDX08sCZgUc9/6MHni9IWuFLm1Z3HTCXu2z9fN62Q==} @@ -3442,6 +3760,13 @@ packages: option@0.2.4: resolution: {integrity: sha512-pkEqbDyl8ou5cpq+VsnQbe/WlEy5qS7xPzMS1U55OCG9KPvwFD46zDbxQIj3egJSFc3D+XhYOPUzz49zQAVy7A==} + oxc-parser@0.155.0: + resolution: {integrity: sha512-C/ReSf1JUFukK/zvk6jhcNibSssvas5+ktIzK7fbKVd3CedWe59C3QwHEOrEZj/4Rt9+xg0Jeis/Jv1F/KM1mw==} + engines: {node: ^20.19.0 || >=22.12.0} + + oxc-resolver@11.24.2: + resolution: {integrity: sha512-FY91FiDBj7ls5MsFS9jN3tjz2o0/zsdSsymlakySaBwVJZorHhkWyICLZMKxlu1R9vYo+sd3z1jwb4J8x7bNDw==} + p-each-series@3.0.0: resolution: {integrity: sha512-lastgtAdoH9YaLyDa5i5z64q+kzOcQHsQ5SsZJD3q0VEyI8mq872S3geuNbRUQLVAE9siMfgKrpj7MloKFHruw==} engines: {node: '>=12'} @@ -3454,6 +3779,10 @@ packages: resolution: {integrity: sha512-37/tPdZ3oJwHaS3gNJdenCDB3Tz26i9sjhnguBtvN0vYlRIiDNnvTWkuh+0hETV9rLPdJ3rlL3yVOYPIAnM8rw==} engines: {node: '>=18'} + p-finally@1.0.0: + resolution: {integrity: sha512-LICb2p9CB7FS+0eR1oqWnHhp0FljGLZCWBE9aix0Uye9W8LTQPwMTYVGWQWIw9RdQiDg4+epXQODwIYJtSJaow==} + engines: {node: '>=4'} + p-limit@1.3.0: resolution: {integrity: sha512-vvcXsLAJ9Dr5rQOPk7toZQZJApBl2K4J6dANSsEuh6QI41JYcsS/qhTGa9ErIUUgK3WNQoJYvylxvjqmiqEA9Q==} engines: {node: '>=4'} @@ -3470,6 +3799,10 @@ packages: resolution: {integrity: sha512-MitaVsCuCFIvOLLPIU7NnfrZvS9H9h7kwMUkDo+T2pEISaJD48IV9S8iIdXB7PsvvdxyYcsSTTrr90XKsbulNw==} engines: {node: '>=18'} + p-queue@6.6.2: + resolution: {integrity: sha512-RwFpb72c/BhQLEXIZ5K2e+AhgNVmIejGlTgiB9MzZ0e93GRvqZ7uSi0dvRF7/XIXDeNkra2fNHBxTyPDGySpjQ==} + engines: {node: '>=8'} + p-queue@9.3.5: resolution: {integrity: sha512-Lki/zhXFVmDhesBUn/yHRLW9Bte7mE7xoFHzTR8sIomV7a2z+LxYAPkGvdfidNiu4fKSfjtawixxSHcZci5QcQ==} engines: {node: '>=20'} @@ -3478,6 +3811,10 @@ packages: resolution: {integrity: sha512-xsrIUgI0Kn6iyDYm9StOpOeK29XM1aboGji26+QEortiFST1hGZaUQOLhtEbqHErPpGW/aSz6allwK2qcptp0Q==} engines: {node: '>=12'} + p-timeout@3.2.0: + resolution: {integrity: sha512-rhIwUycgwwKcP9yTOOFK/AKsAopjjCakVqLHePO3CC6Mir1Z99xT+R63jZxAT5lFZLa2inS5h+ZS2GvR99/FBg==} + engines: {node: '>=8'} + p-timeout@6.1.4: resolution: {integrity: sha512-MyIV3ZA/PmyBN/ud8vV9XzwTrNtR4jFrObymZYnZqMmW0zA8Z17vnT0rBgFE/TlohB+YCHqXMgZzb3Csp49vqg==} engines: {node: '>=14.16'} @@ -3957,6 +4294,10 @@ packages: resolution: {integrity: sha512-4gB8na07fecVVkOI6Rs4e7T6NOTki5EmL7TUduTs6bu3EdnSycntVJ4re8kgZA+wx9IueI2Y11bfbgwtzuE0KQ==} engines: {node: '>=0.10.0'} + strip-json-comments@5.0.3: + resolution: {integrity: sha512-1tB5mhVo7U+ETBKNf92xT4hrQa3pm0MZ0PQvuDnWgAAGHDsfp4lPSpiS6psrSiet87wyGPh9ft6wmhOMQ0hDiw==} + engines: {node: '>=14.16'} + super-regex@1.1.0: resolution: {integrity: sha512-WHkws2ZflZe41zj6AolvvmaTrWds/VuyeYr9iPVv/oQeaIoVxMKaushfFWpOGDT+GuBrM/sVqF8KUCYQlSSTdQ==} engines: {node: '>=18'} @@ -4128,6 +4469,10 @@ packages: ultrahtml@1.7.0: resolution: {integrity: sha512-2xRd0VHoAQE4M+vF/DvFFB7pUV0ZxTW1TLi7lHQWnF/Sb5TPeEUV/l+hxcNnGO00ZXGnR0voCMmYRKQf+rvJ2g==} + unbash@6.0.2: + resolution: {integrity: sha512-UhcFfg9u8OpFPnUqdXSKgbGHwk2iEHaEePfpHylako6i4rpdmfoOgCVBBlou4WugaFfNunIHby01mKssJzVTCA==} + engines: {node: '>=14'} + uncrypto@0.1.3: resolution: {integrity: sha512-Ql87qFHB3s/De2ClA9e0gsnS6zXG27SkTiSJwjCc9MebbfapQfuPzumMIUMi38ezPZVNFcHI9sUIepeQfw8J8Q==} @@ -4503,6 +4848,10 @@ packages: vscode-uri@3.2.0: resolution: {integrity: sha512-m2gXo3bn0G1kT9InzMf07fTbqMbGtyckj3bH5ktLO+1Ssv+yiATZ4dhwaQv9UZWxJh6E9IFGnQyjgWVDWVBDrg==} + walk-up-path@4.0.0: + resolution: {integrity: sha512-3hu+tD8YzSLGuFYtPRb48vdhKMi0KQV5sn+uWr8+7dMEq/2G/dtLrdDinkLjqq5TIbIBjYJ4Ax/n3YiaW7QM8A==} + engines: {node: 20 || >=22} + web-worker@1.5.0: resolution: {integrity: sha512-RiMReJrTAiA+mBjGONMnjVDP2u3p9R1vkcGz6gDIrOMT3oGuYwX2WRMYI9ipkphSuE5XKEhydbhNEJh4NY9mlw==} @@ -4882,6 +5231,8 @@ snapshots: dependencies: fontkitten: 1.0.3 + '@cfworker/json-schema@4.1.4': {} + '@clack/core@1.5.1': dependencies: fast-wrap-ansi: 0.2.2 @@ -5043,6 +5394,12 @@ snapshots: tslib: 2.8.1 optional: true + '@emnapi/core@1.11.2': + dependencies: + '@emnapi/wasi-threads': 1.2.2 + tslib: 2.8.1 + optional: true + '@emnapi/core@1.11.3': dependencies: '@emnapi/wasi-threads': 1.2.3 @@ -5053,6 +5410,11 @@ snapshots: tslib: 2.8.1 optional: true + '@emnapi/runtime@1.11.2': + dependencies: + tslib: 2.8.1 + optional: true + '@emnapi/runtime@1.11.3': dependencies: tslib: 2.8.1 @@ -5322,11 +5684,32 @@ snapshots: '@lancedb/lancedb-win32-arm64-msvc': 0.30.0 '@lancedb/lancedb-win32-x64-msvc': 0.30.0 + '@langchain/core@1.3.2(@opentelemetry/api@1.10.0)': + dependencies: + '@cfworker/json-schema': 4.1.4 + '@standard-schema/spec': 1.1.0 + js-tiktoken: 1.0.21 + langsmith: 0.11.6(@opentelemetry/api@1.10.0) + mustache: 4.2.0 + p-queue: 6.6.2 + zod: 4.6.7 + transitivePeerDependencies: + - '@opentelemetry/api' + - '@opentelemetry/exporter-trace-otlp-proto' + - '@opentelemetry/sdk-trace-base' + - openai + - ws + + '@langchain/textsplitters@1.0.2(@langchain/core@1.3.2(@opentelemetry/api@1.10.0))': + dependencies: + '@langchain/core': 1.3.2(@opentelemetry/api@1.10.0) + js-tiktoken: 1.0.21 + '@lucide/astro@1.61.0(astro@7.4.2(@types/node@22.20.5)(esbuild@0.28.2)(jiti@2.7.0)(yaml@2.9.1))': dependencies: astro: 7.4.2(@types/node@22.20.5)(esbuild@0.28.2)(jiti@2.7.0)(yaml@2.9.1) - '@modelcontextprotocol/sdk@1.32.2(supports-color@7.2.0)(zod@4.6.7)': + '@modelcontextprotocol/sdk@1.32.2(@cfworker/json-schema@4.1.4)(supports-color@7.2.0)(zod@4.6.7)': dependencies: '@hono/node-server': 2.1.5(hono@4.13.14) ajv: 8.20.0 @@ -5345,6 +5728,8 @@ snapshots: raw-body: 3.0.2 zod: 4.6.7 zod-to-json-schema: 3.25.2(zod@4.6.7) + optionalDependencies: + '@cfworker/json-schema': 4.1.4 transitivePeerDependencies: - supports-color @@ -5361,6 +5746,13 @@ snapshots: '@tybys/wasm-util': 0.10.4 optional: true + '@napi-rs/wasm-runtime@1.2.6(@emnapi/core@1.11.2)(@emnapi/runtime@1.11.2)': + dependencies: + '@emnapi/core': 1.11.2 + '@emnapi/runtime': 1.11.2 + '@tybys/wasm-util': 0.10.4 + optional: true + '@napi-rs/wasm-runtime@1.2.6(@emnapi/core@1.11.3)(@emnapi/runtime@1.11.3)': dependencies: '@emnapi/core': 1.11.3 @@ -5449,10 +5841,130 @@ snapshots: '@opentelemetry/api@1.10.0': optional: true + '@orama/orama@3.1.18': {} + '@oslojs/encoding@1.1.0': {} + '@oxc-parser/binding-android-arm-eabi@0.155.0': + optional: true + + '@oxc-parser/binding-android-arm64@0.155.0': + optional: true + + '@oxc-parser/binding-darwin-arm64@0.155.0': + optional: true + + '@oxc-parser/binding-darwin-x64@0.155.0': + optional: true + + '@oxc-parser/binding-freebsd-x64@0.155.0': + optional: true + + '@oxc-parser/binding-linux-arm-gnueabihf@0.155.0': + optional: true + + '@oxc-parser/binding-linux-arm-musleabihf@0.155.0': + optional: true + + '@oxc-parser/binding-linux-arm64-gnu@0.155.0': + optional: true + + '@oxc-parser/binding-linux-arm64-musl@0.155.0': + optional: true + + '@oxc-parser/binding-linux-ppc64-gnu@0.155.0': + optional: true + + '@oxc-parser/binding-linux-riscv64-gnu@0.155.0': + optional: true + + '@oxc-parser/binding-linux-riscv64-musl@0.155.0': + optional: true + + '@oxc-parser/binding-linux-s390x-gnu@0.155.0': + optional: true + + '@oxc-parser/binding-linux-x64-gnu@0.155.0': + optional: true + + '@oxc-parser/binding-linux-x64-musl@0.155.0': + optional: true + + '@oxc-parser/binding-openharmony-arm64@0.155.0': + optional: true + + '@oxc-parser/binding-win32-arm64-msvc@0.155.0': + optional: true + + '@oxc-parser/binding-win32-ia32-msvc@0.155.0': + optional: true + + '@oxc-parser/binding-win32-x64-msvc@0.155.0': + optional: true + '@oxc-project/types@0.155.0': {} + '@oxc-resolver/binding-android-arm-eabi@11.24.2': + optional: true + + '@oxc-resolver/binding-android-arm64@11.24.2': + optional: true + + '@oxc-resolver/binding-darwin-arm64@11.24.2': + optional: true + + '@oxc-resolver/binding-darwin-x64@11.24.2': + optional: true + + '@oxc-resolver/binding-freebsd-x64@11.24.2': + optional: true + + '@oxc-resolver/binding-linux-arm-gnueabihf@11.24.2': + optional: true + + '@oxc-resolver/binding-linux-arm-musleabihf@11.24.2': + optional: true + + '@oxc-resolver/binding-linux-arm64-gnu@11.24.2': + optional: true + + '@oxc-resolver/binding-linux-arm64-musl@11.24.2': + optional: true + + '@oxc-resolver/binding-linux-ppc64-gnu@11.24.2': + optional: true + + '@oxc-resolver/binding-linux-riscv64-gnu@11.24.2': + optional: true + + '@oxc-resolver/binding-linux-riscv64-musl@11.24.2': + optional: true + + '@oxc-resolver/binding-linux-s390x-gnu@11.24.2': + optional: true + + '@oxc-resolver/binding-linux-x64-gnu@11.24.2': + optional: true + + '@oxc-resolver/binding-linux-x64-musl@11.24.2': + optional: true + + '@oxc-resolver/binding-openharmony-arm64@11.24.2': + optional: true + + '@oxc-resolver/binding-wasm32-wasi@11.24.2': + dependencies: + '@emnapi/core': 1.11.2 + '@emnapi/runtime': 1.11.2 + '@napi-rs/wasm-runtime': 1.2.6(@emnapi/core@1.11.2)(@emnapi/runtime@1.11.2) + optional: true + + '@oxc-resolver/binding-win32-arm64-msvc@11.24.2': + optional: true + + '@oxc-resolver/binding-win32-x64-msvc@11.24.2': + optional: true + '@pnpm/config.env-replace@1.1.0': {} '@pnpm/network.ca-file@1.0.2': @@ -5816,6 +6328,8 @@ snapshots: '@sindresorhus/merge-streams@4.0.0': {} + '@standard-schema/spec@1.1.0': {} + '@swc/helpers@0.5.23': dependencies: tslib: 2.8.1 @@ -6030,7 +6544,7 @@ snapshots: obug: 3.0.0 std-env: 4.3.0 tinyrainbow: 3.2.0 - vitest: 5.0.5(@opentelemetry/api@1.10.0)(@types/node@22.20.5)(@vitest/coverage-v8@5.0.5)(vite@8.3.5(@types/node@22.20.5)(esbuild@0.28.2)(jiti@2.7.0)(yaml@2.9.1)) + vitest: 5.0.5(@opentelemetry/api@1.10.0)(@types/node@24.19.2)(@vitest/coverage-v8@5.0.5)(vite@8.3.5(@types/node@24.19.2)(esbuild@0.28.2)(jiti@2.7.0)(yaml@2.9.1)) '@vitest/istanbul-lib-coverage@1.0.2': {} @@ -6047,6 +6561,15 @@ snapshots: optionalDependencies: vite: 8.3.5(@types/node@22.20.5)(esbuild@0.28.2)(jiti@2.7.0)(yaml@2.9.1) + '@vitest/mocker@5.0.5(vite@8.3.5(@types/node@24.19.2)(esbuild@0.28.2)(jiti@2.7.0)(yaml@2.9.1))': + dependencies: + '@jridgewell/trace-mapping': 0.3.31 + '@vitest/spy': 5.0.5 + estree-walker: 3.0.3 + magic-string: 1.4.4 + optionalDependencies: + vite: 8.3.5(@types/node@24.19.2)(esbuild@0.28.2)(jiti@2.7.0)(yaml@2.9.1) + '@vitest/spy@5.0.5': {} '@volar/kit@2.4.30(typescript@6.0.3)': @@ -6747,6 +7270,8 @@ snapshots: etag@1.8.1: {} + eventemitter3@4.0.7: {} + eventemitter3@5.0.4: {} eventsource-parser@3.1.1: {} @@ -6853,6 +7378,10 @@ snapshots: dependencies: reusify: 1.1.0 + fd-package-json@2.0.0: + dependencies: + walk-up-path: 4.0.0 + fdir@6.5.0(picomatch@4.0.7): optionalDependencies: picomatch: 4.0.7 @@ -6913,6 +7442,11 @@ snapshots: dependencies: tiny-inflate: 1.0.3 + formatly@0.11.4: + dependencies: + fd-package-json: 2.0.0 + package-manager-detector: 1.9.0 + forwarded@0.2.0: {} fresh@2.0.0: {} @@ -6961,6 +7495,10 @@ snapshots: '@sec-ant/readable-stream': 0.4.1 is-stream: 4.0.1 + get-tsconfig@4.14.3: + dependencies: + resolve-pkg-maps: 1.0.0 + get-tsconfig@5.0.0-beta.6: dependencies: resolve-pkg-maps: 1.0.0 @@ -7204,6 +7742,10 @@ snapshots: jose@6.2.13: {} + js-tiktoken@1.0.21: + dependencies: + base64-js: 1.5.1 + js-tokens@10.0.0: {} js-tokens@4.0.0: {} @@ -7243,6 +7785,27 @@ snapshots: kleur@4.1.5: {} + knip@6.44.1: + dependencies: + formatly: 0.11.4 + get-tsconfig: 4.14.3 + jiti: 2.7.0 + oxc-parser: 0.155.0 + oxc-resolver: 11.24.2 + picomatch: 4.0.7 + smol-toml: 1.9.1 + strip-json-comments: 5.0.3 + tinyglobby: 0.2.17 + unbash: 6.0.2 + yaml: 2.9.1 + zod: 4.6.7 + + langsmith@0.11.6(@opentelemetry/api@1.10.0): + dependencies: + p-queue: 6.6.2 + optionalDependencies: + '@opentelemetry/api': 1.10.0 + leac@0.7.0: {} lie@3.3.0: @@ -7503,6 +8066,8 @@ snapshots: minimist@1.2.8: {} + minisearch@7.2.0: {} + mri@1.2.0: {} mrmime@2.0.1: {} @@ -7511,6 +8076,8 @@ snapshots: muggle-string@0.4.1: {} + mustache@4.2.0: {} + mz@2.7.0: dependencies: any-promise: 1.3.0 @@ -7632,6 +8199,52 @@ snapshots: option@0.2.4: {} + oxc-parser@0.155.0: + dependencies: + '@oxc-project/types': 0.155.0 + optionalDependencies: + '@oxc-parser/binding-android-arm-eabi': 0.155.0 + '@oxc-parser/binding-android-arm64': 0.155.0 + '@oxc-parser/binding-darwin-arm64': 0.155.0 + '@oxc-parser/binding-darwin-x64': 0.155.0 + '@oxc-parser/binding-freebsd-x64': 0.155.0 + '@oxc-parser/binding-linux-arm-gnueabihf': 0.155.0 + '@oxc-parser/binding-linux-arm-musleabihf': 0.155.0 + '@oxc-parser/binding-linux-arm64-gnu': 0.155.0 + '@oxc-parser/binding-linux-arm64-musl': 0.155.0 + '@oxc-parser/binding-linux-ppc64-gnu': 0.155.0 + '@oxc-parser/binding-linux-riscv64-gnu': 0.155.0 + '@oxc-parser/binding-linux-riscv64-musl': 0.155.0 + '@oxc-parser/binding-linux-s390x-gnu': 0.155.0 + '@oxc-parser/binding-linux-x64-gnu': 0.155.0 + '@oxc-parser/binding-linux-x64-musl': 0.155.0 + '@oxc-parser/binding-openharmony-arm64': 0.155.0 + '@oxc-parser/binding-win32-arm64-msvc': 0.155.0 + '@oxc-parser/binding-win32-ia32-msvc': 0.155.0 + '@oxc-parser/binding-win32-x64-msvc': 0.155.0 + + oxc-resolver@11.24.2: + optionalDependencies: + '@oxc-resolver/binding-android-arm-eabi': 11.24.2 + '@oxc-resolver/binding-android-arm64': 11.24.2 + '@oxc-resolver/binding-darwin-arm64': 11.24.2 + '@oxc-resolver/binding-darwin-x64': 11.24.2 + '@oxc-resolver/binding-freebsd-x64': 11.24.2 + '@oxc-resolver/binding-linux-arm-gnueabihf': 11.24.2 + '@oxc-resolver/binding-linux-arm-musleabihf': 11.24.2 + '@oxc-resolver/binding-linux-arm64-gnu': 11.24.2 + '@oxc-resolver/binding-linux-arm64-musl': 11.24.2 + '@oxc-resolver/binding-linux-ppc64-gnu': 11.24.2 + '@oxc-resolver/binding-linux-riscv64-gnu': 11.24.2 + '@oxc-resolver/binding-linux-riscv64-musl': 11.24.2 + '@oxc-resolver/binding-linux-s390x-gnu': 11.24.2 + '@oxc-resolver/binding-linux-x64-gnu': 11.24.2 + '@oxc-resolver/binding-linux-x64-musl': 11.24.2 + '@oxc-resolver/binding-openharmony-arm64': 11.24.2 + '@oxc-resolver/binding-wasm32-wasi': 11.24.2 + '@oxc-resolver/binding-win32-arm64-msvc': 11.24.2 + '@oxc-resolver/binding-win32-x64-msvc': 11.24.2 + p-each-series@3.0.0: {} p-event@6.0.1: @@ -7642,6 +8255,8 @@ snapshots: dependencies: p-map: 7.0.8 + p-finally@1.0.0: {} + p-limit@1.3.0: dependencies: p-try: 1.0.0 @@ -7656,6 +8271,11 @@ snapshots: p-map@7.0.8: {} + p-queue@6.6.2: + dependencies: + eventemitter3: 4.0.7 + p-timeout: 3.2.0 + p-queue@9.3.5: dependencies: eventemitter3: 5.0.4 @@ -7663,6 +8283,10 @@ snapshots: p-reduce@3.0.0: {} + p-timeout@3.2.0: + dependencies: + p-finally: 1.0.0 + p-timeout@6.1.4: {} p-timeout@7.0.2: {} @@ -8232,6 +8856,8 @@ snapshots: strip-json-comments@2.0.1: {} + strip-json-comments@5.0.3: {} + super-regex@1.1.0: dependencies: function-timeout: 1.0.2 @@ -8393,6 +9019,8 @@ snapshots: ultrahtml@1.7.0: {} + unbash@6.0.2: {} + uncrypto@0.1.3: {} underscore@1.13.8: {} @@ -8520,6 +9148,20 @@ snapshots: jiti: 2.7.0 yaml: 2.9.1 + vite@8.3.5(@types/node@24.19.2)(esbuild@0.28.2)(jiti@2.7.0)(yaml@2.9.1): + dependencies: + lightningcss: 1.33.0 + picomatch: 4.0.7 + postcss: 8.5.31 + rolldown: 1.2.15 + tinyglobby: 0.2.17 + optionalDependencies: + '@types/node': 24.19.2 + esbuild: 0.28.2 + fsevents: 2.3.3 + jiti: 2.7.0 + yaml: 2.9.1 + vitefu@1.1.3(vite@8.3.5(@types/node@22.20.5)(esbuild@0.28.2)(jiti@2.7.0)(yaml@2.9.1)): optionalDependencies: vite: 8.3.5(@types/node@22.20.5)(esbuild@0.28.2)(jiti@2.7.0)(yaml@2.9.1) @@ -8547,6 +9189,29 @@ snapshots: transitivePeerDependencies: - msw + vitest@5.0.5(@opentelemetry/api@1.10.0)(@types/node@24.19.2)(@vitest/coverage-v8@5.0.5)(vite@8.3.5(@types/node@24.19.2)(esbuild@0.28.2)(jiti@2.7.0)(yaml@2.9.1)): + dependencies: + '@types/chai': 5.2.3 + '@vitest/mocker': 5.0.5(vite@8.3.5(@types/node@24.19.2)(esbuild@0.28.2)(jiti@2.7.0)(yaml@2.9.1)) + chai: 6.3.0 + es-module-lexer: 2.3.2 + expect-type: 1.4.0 + magic-string: 1.4.4 + obug: 3.0.0 + picomatch: 4.0.7 + std-env: 4.3.0 + tinybench: 6.2.2 + tinyexec: 1.3.1 + tinyglobby: 0.2.17 + vite: 8.3.5(@types/node@24.19.2)(esbuild@0.28.2)(jiti@2.7.0)(yaml@2.9.1) + why-is-node-running: 3.2.1 + optionalDependencies: + '@opentelemetry/api': 1.10.0 + '@types/node': 24.19.2 + '@vitest/coverage-v8': 5.0.5(vitest@5.0.5) + transitivePeerDependencies: + - msw + volar-service-css@0.0.71(@volar/language-service@2.4.30(typescript@6.0.3)): dependencies: vscode-css-languageservice: 6.3.10 @@ -8655,6 +9320,8 @@ snapshots: vscode-uri@3.2.0: {} + walk-up-path@4.0.0: {} + web-worker@1.5.0: {} which@2.0.2: From d9ab66b5d723ae92a2b7086f2b6ab2718d58aaa4 Mon Sep 17 00:00:00 2001 From: Jean-Baptiste THERY Date: Sun, 1 Nov 2026 19:17:54 +0700 Subject: [PATCH 4/5] feat(landing): add GitHub and npm hero buttons and a sectioned mobile drawer - add View on GitHub and View on npm buttons under the hero calls to action - rebuild the mobile drawer as cards of icon rows with chevrons, Documentation and Project sections, language pills, the version, and pinned GitHub and npm buttons - list every project link from one module shared by the hero, drawer, footer, and structured data, adding the FAQ, benchmarks, and AGPL license links that some surfaces missed - show the measured 39 ms median on the 29,778-chunk monorepo in the hero proof --- packages/ragmir-landing/messages/en.json | 11 +- packages/ragmir-landing/messages/fr.json | 11 +- .../src/components/landing-navbar.tsx | 295 +++++++++++++++--- .../src/components/npm-icon.tsx | 15 + .../src/components/sections/agents.astro | 3 +- .../src/components/sections/closing-cta.astro | 5 +- .../src/components/sections/footer.astro | 21 +- .../src/components/sections/hero.astro | 25 +- .../ragmir-landing/src/lib/project-links.ts | 44 +++ .../src/pages/[...locale]/index.astro | 15 +- .../src/pages/[...locale]/team.astro | 2 + .../src/public-contract.test.ts | 51 ++- 12 files changed, 424 insertions(+), 74 deletions(-) create mode 100644 packages/ragmir-landing/src/components/npm-icon.tsx create mode 100644 packages/ragmir-landing/src/lib/project-links.ts diff --git a/packages/ragmir-landing/messages/en.json b/packages/ragmir-landing/messages/en.json index 45248b6..566baa6 100644 --- a/packages/ragmir-landing/messages/en.json +++ b/packages/ragmir-landing/messages/en.json @@ -4,10 +4,13 @@ "nav_agents": "Work modes", "nav_use_cases": "Use cases", "nav_team": "Team", + "nav_faq": "FAQ", "nav_menu": "Menu", "nav_menu_close": "Close menu", "nav_github": "GitHub", "nav_aria_label": "Main navigation", + "nav_section_docs": "Documentation", + "nav_section_project": "Project", "language_label": "Language", "seo_home_title": "Ragmir | Agentic RAG for developers", "seo_home_description": "Open-source agentic RAG with local document search, exact citations, and MCP. Connect your agent or chat through TypeScript or CLI, with your choice of model.", @@ -22,11 +25,13 @@ "hero_description": "Give your agent precise context from code, specs, Office files, and PDFs. Ragmir indexes locally and returns exact citations. Your agent searches, refines, and answers with the model you choose.", "hero_primary_cta": "Install Ragmir", "hero_secondary_cta": "See use cases", + "hero_github_cta": "View on GitHub", + "hero_npm_cta": "View on npm", "hero_npm_downloads_label": "Ragmir downloads/month on npm", "hero_proof_recall_value": "93%", "hero_proof_recall_label": "of real developer questions find the right file in the top 10", - "hero_proof_speed_value": "40 ms", - "hero_proof_speed_label": "median search on a 43,874-chunk monorepo", + "hero_proof_speed_value": "39 ms", + "hero_proof_speed_label": "median search on a 29,778-chunk monorepo", "hero_proof_link": "See the benchmarks", "hero_metric_private_value": "Fewer tokens", "hero_metric_private_label": "Cited passages, never the same one twice.", @@ -177,7 +182,9 @@ "footer_link_releases": "Releases", "footer_link_changelog": "Changelog", "footer_link_api": "API reference", + "footer_link_benchmarks": "Benchmarks", "footer_link_commercial_license": "Commercial license", + "footer_link_license": "AGPL-3.0 license", "footer_text": "Ragmir is JCode Labs’ open-source retrieval layer for agentic RAG: project documents, exact citations, and your choice of agent and model.", "footer_version_label": "Current Ragmir version on npm", "strengths_eyebrow": "Useful by design", diff --git a/packages/ragmir-landing/messages/fr.json b/packages/ragmir-landing/messages/fr.json index e2cba1c..9ef3bc6 100644 --- a/packages/ragmir-landing/messages/fr.json +++ b/packages/ragmir-landing/messages/fr.json @@ -4,10 +4,13 @@ "nav_agents": "Modes", "nav_use_cases": "Cas d'usage", "nav_team": "Équipe", + "nav_faq": "FAQ", "nav_menu": "Menu", "nav_menu_close": "Fermer le menu", "nav_github": "GitHub", "nav_aria_label": "Navigation principale", + "nav_section_docs": "Documentation", + "nav_section_project": "Projet", "language_label": "Langue", "seo_home_title": "Ragmir | Le RAG agentique pour développeurs", "seo_home_description": "RAG agentique open source : recherche documentaire locale, citations précises et MCP. Connectez votre agent ou chat via TypeScript ou CLI, avec le modèle de votre choix.", @@ -22,11 +25,13 @@ "hero_description": "Donnez à votre agent le contexte précis du code, des specs, des fichiers Office et des PDF. Ragmir indexe en local et cite ses sources. Votre agent cherche, affine et répond avec le modèle de votre choix.", "hero_primary_cta": "Installer Ragmir", "hero_secondary_cta": "Voir les cas d'usage", + "hero_github_cta": "Voir sur GitHub", + "hero_npm_cta": "Voir sur npm", "hero_npm_downloads_label": "téléchargements/mois de Ragmir sur npm", "hero_proof_recall_value": "93 %", "hero_proof_recall_label": "des vraies questions de développeurs trouvent le bon fichier dans le top 10", - "hero_proof_speed_value": "40 ms", - "hero_proof_speed_label": "de recherche médiane sur un monorepo de 43 874 passages", + "hero_proof_speed_value": "39 ms", + "hero_proof_speed_label": "de recherche médiane sur un monorepo de 29 778 passages", "hero_proof_link": "Voir les benchmarks", "hero_metric_private_value": "Moins de tokens", "hero_metric_private_label": "Des passages cités, jamais deux fois le même.", @@ -177,7 +182,9 @@ "footer_link_releases": "Versions", "footer_link_changelog": "Changelog", "footer_link_api": "Référence API", + "footer_link_benchmarks": "Benchmarks", "footer_link_commercial_license": "Licence commerciale", + "footer_link_license": "Licence AGPL-3.0", "footer_text": "Ragmir est la bibliothèque open source de JCode Labs pour le RAG agentique : documents projet, citations précises, agent et modèle de votre choix.", "footer_version_label": "Version actuelle de Ragmir sur npm", "strengths_eyebrow": "Pensé pour être utile", diff --git a/packages/ragmir-landing/src/components/landing-navbar.tsx b/packages/ragmir-landing/src/components/landing-navbar.tsx index 25e1a3d..e0f942c 100644 --- a/packages/ragmir-landing/src/components/landing-navbar.tsx +++ b/packages/ragmir-landing/src/components/landing-navbar.tsx @@ -1,7 +1,36 @@ -import { ChevronDown, Globe2, Menu, X } from "lucide-react" -import { useCallback, useEffect, useRef, useState } from "react" +import { + ArrowUpRight, + BookOpen, + Bot, + Briefcase, + ChevronDown, + ChevronRight, + CircleHelp, + Download, + FileCode, + Gauge, + Globe2, + History, + Layers, + type LucideIcon, + Menu, + Scale, + Sparkles, + Tag, + Users, + X, +} from "lucide-react" +import { useCallback, useEffect, useId, useRef, useState } from "react" +import { + GITHUB_URL, + NPM_URL, + PROJECT_LINK_LABEL_KEYS, + PROJECT_LINKS, + type ProjectLinkKey, +} from "../lib/project-links" import { cn } from "../lib/utils" import { GithubIcon } from "./github-icon" +import { NpmIcon } from "./npm-icon" import { Button } from "./ui/button" interface AlternateLocale { @@ -19,7 +48,10 @@ interface LandingNavbarProps { localizedLibraryUrl: string localizedFeaturesUrl: string localizedAgentsUrl: string + localizedFaqUrl: string localizedTeamUrl: string + /** Path of the page on screen, so the drawer can light its own row. */ + currentPath?: string } const externalLinkProps = { @@ -27,7 +59,120 @@ const externalLinkProps = { rel: "noopener noreferrer", } as const -const GITHUB_URL = "https://github.com/jcode-works/jcode-ragmir" +interface DrawerLink { + href: string + label: string + icon: LucideIcon + external?: boolean +} + +/** The drawer's sections under the pages card: documentation first, then the project's records. */ +const DRAWER_SECTIONS: ReadonlyArray<{ + titleKey: string + links: ReadonlyArray<{ key: ProjectLinkKey; icon: LucideIcon }> +}> = [ + { + titleKey: "nav_section_docs", + links: [ + { key: "docs", icon: BookOpen }, + { key: "apiReference", icon: FileCode }, + { key: "benchmarks", icon: Gauge }, + ], + }, + { + titleKey: "nav_section_project", + links: [ + { key: "releases", icon: Tag }, + { key: "changelog", icon: History }, + { key: "license", icon: Scale }, + { key: "commercialLicense", icon: Briefcase }, + ], + }, +] + +/** One drawer row: a grey icon, the label, then a chevron, or an arrow for a link that leaves the site. */ +const DRAWER_ROW_CLASS_NAME = + "flex min-h-11 items-center gap-3 px-4 font-semibold text-foreground text-sm transition-colors hover:bg-foreground/[0.04] focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-ring focus-visible:ring-inset active:bg-foreground/10" + +const DRAWER_SECTION_TITLE_CLASS_NAME = + "mb-2 font-black text-foreground text-sm uppercase tracking-tight" + +/** One card of the drawer: rows one under the other, the page on screen lit. */ +function DrawerLinkList({ + links, + currentPath, + onNavigate, + tabIndex, +}: { + links: readonly DrawerLink[] + currentPath: string | undefined + onNavigate: () => void + tabIndex: number +}): React.JSX.Element { + return ( + + ) +} + +/** One titled group of the drawer: the section title over a card of rows. */ +function DrawerSection({ + title, + links, + currentPath, + onNavigate, + tabIndex, +}: { + title: string + links: readonly DrawerLink[] + currentPath: string | undefined + onNavigate: () => void + tabIndex: number +}): React.JSX.Element { + const titleId = useId() + return ( +
+

+ {title} +

+ +
+ ) +} export function RagmirLogo(): React.JSX.Element { return ( @@ -102,7 +247,9 @@ export function LandingNavbar({ localizedLibraryUrl, localizedFeaturesUrl, localizedAgentsUrl, + localizedFaqUrl, localizedTeamUrl, + currentPath, }: LandingNavbarProps): React.JSX.Element { const t = (key: string): string => translations[key] ?? key const version = import.meta.env.PUBLIC_RAGMIR_VERSION?.trim() @@ -121,6 +268,27 @@ export function LandingNavbar({ { href: localizedTeamUrl, label: t("nav_team") }, ] + const drawerPageLinks: DrawerLink[] = [ + { href: localizedUseCasesUrl, label: t("nav_use_cases"), icon: Layers }, + { href: localizedLibraryUrl, label: t("nav_library"), icon: Download }, + { href: localizedAgentsUrl, label: t("nav_agents"), icon: Bot }, + { href: localizedFeaturesUrl, label: t("nav_features"), icon: Sparkles }, + { href: localizedFaqUrl, label: t("nav_faq"), icon: CircleHelp }, + { href: localizedTeamUrl, label: t("nav_team"), icon: Users }, + ] + const drawerSections = DRAWER_SECTIONS.map((section) => ({ + title: t(section.titleKey), + links: section.links.map( + ({ key, icon }): DrawerLink => ({ + href: PROJECT_LINKS[key], + label: t(PROJECT_LINK_LABEL_KEYS[key]), + icon, + external: true, + }), + ), + })) + const drawerTabIndex = menuOpen ? 0 : -1 + const closeMenu = useCallback(() => { menuTriggerRef.current?.focus() setMenuOpen(false) @@ -200,7 +368,7 @@ export function LandingNavbar({ {version && ( + + + - diff --git a/packages/ragmir-landing/src/components/npm-icon.tsx b/packages/ragmir-landing/src/components/npm-icon.tsx new file mode 100644 index 0000000..77dd225 --- /dev/null +++ b/packages/ragmir-landing/src/components/npm-icon.tsx @@ -0,0 +1,15 @@ +// npm mark from Simple Icons (CC0). lucide-react ships no brand icons, so it is inlined here as an +// SVG and colored via currentColor, like the GitHub mark. +export function NpmIcon(props: React.SVGProps): React.JSX.Element { + return ( + + ) +} diff --git a/packages/ragmir-landing/src/components/sections/agents.astro b/packages/ragmir-landing/src/components/sections/agents.astro index 631d2bc..271279c 100644 --- a/packages/ragmir-landing/src/components/sections/agents.astro +++ b/packages/ragmir-landing/src/components/sections/agents.astro @@ -1,6 +1,7 @@ --- import { BookOpen, Cloud, Cpu, Server } from "@lucide/astro" import { CommandCopyBox } from "../command-copy" +import { GITHUB_URL } from "../../lib/project-links" interface Props { translations: Record @@ -46,7 +47,7 @@ const operatingModes = [

{t("agents_model_title")}

{t("agents_model_text")}

- + {t("agents_model_link")} diff --git a/packages/ragmir-landing/src/components/sections/closing-cta.astro b/packages/ragmir-landing/src/components/sections/closing-cta.astro index 2b056ea..e2143bf 100644 --- a/packages/ragmir-landing/src/components/sections/closing-cta.astro +++ b/packages/ragmir-landing/src/components/sections/closing-cta.astro @@ -1,5 +1,6 @@ --- import { ArrowRight } from "@lucide/astro" +import { GITHUB_URL } from "../../lib/project-links" interface Props { translations: Record @@ -36,12 +37,12 @@ const t = (key: string): string => translations[key] ?? key - {t("closing_primary_cta")} | github.com/jcode-works/jcode-ragmir + {t("closing_primary_cta")} | {GITHUB_URL.replace(/^https:\/\//u, "")} diff --git a/packages/ragmir-landing/src/components/sections/footer.astro b/packages/ragmir-landing/src/components/sections/footer.astro index f8379ab..2ef761e 100644 --- a/packages/ragmir-landing/src/components/sections/footer.astro +++ b/packages/ragmir-landing/src/components/sections/footer.astro @@ -1,6 +1,12 @@ --- import RagmirLogo from "../ragmir-logo.astro" import { RagmirBackground } from "../ragmir-background" +import { + FOOTER_LINK_KEYS, + NPM_URL, + PROJECT_LINK_LABEL_KEYS, + PROJECT_LINKS, +} from "../../lib/project-links" interface Props { translations: Record @@ -11,15 +17,10 @@ const { translations, localizedHomeUrl } = Astro.props const t = (key: string): string => translations[key] ?? key const version = import.meta.env.PUBLIC_RAGMIR_VERSION?.trim() -const footerLinks = [ - { href: "https://github.com/jcode-works/jcode-ragmir", label: t("nav_github") }, - { href: "https://www.npmjs.com/package/@jcode.labs/ragmir", label: t("footer_link_npm") }, - { href: "https://github.com/jcode-works/jcode-ragmir/wiki", label: t("footer_link_docs") }, - { href: "https://github.com/jcode-works/jcode-ragmir/releases/latest", label: t("footer_link_releases") }, - { href: "https://github.com/jcode-works/jcode-ragmir/blob/main/CHANGELOG.md", label: t("footer_link_changelog") }, - { href: "https://github.com/jcode-works/jcode-ragmir/blob/main/docs/api-reference.md", label: t("footer_link_api") }, - { href: "https://github.com/jcode-works/jcode-ragmir/blob/main/COMMERCIAL-LICENSE.md", label: t("footer_link_commercial_license") }, -] +const footerLinks = FOOTER_LINK_KEYS.map((key) => ({ + href: PROJECT_LINKS[key], + label: t(PROJECT_LINK_LABEL_KEYS[key]), +})) const marqueeCopy = "whitespace-nowrap pr-8 font-black text-5xl text-foreground/90 uppercase italic leading-[1.1] tracking-tighter md:pr-14 md:text-7xl" @@ -76,7 +77,7 @@ const marqueeCopy =

{t("footer_text")}

{version && ( +
+ + + {t("hero_github_cta")} + + + + {t("hero_npm_cta")} + +
{monthlyNpmDownloads !== null && (
{t("hero_proof_link")} diff --git a/packages/ragmir-landing/src/lib/project-links.ts b/packages/ragmir-landing/src/lib/project-links.ts new file mode 100644 index 0000000..9c0b872 --- /dev/null +++ b/packages/ragmir-landing/src/lib/project-links.ts @@ -0,0 +1,44 @@ +// Public project destinations. The hero, the mobile drawer, and the footer read this one list, so +// a link added or moved here reaches every surface at once. +export const GITHUB_URL = "https://github.com/jcode-works/jcode-ragmir" +export const NPM_URL = "https://www.npmjs.com/package/@jcode.labs/ragmir" + +export const PROJECT_LINKS = { + github: GITHUB_URL, + npm: NPM_URL, + docs: `${GITHUB_URL}/wiki`, + apiReference: `${GITHUB_URL}/blob/main/docs/api-reference.md`, + benchmarks: `${GITHUB_URL}#benchmarks`, + releases: `${GITHUB_URL}/releases/latest`, + changelog: `${GITHUB_URL}/blob/main/CHANGELOG.md`, + license: `${GITHUB_URL}/blob/main/LICENSE`, + commercialLicense: `${GITHUB_URL}/blob/main/COMMERCIAL-LICENSE.md`, +} as const + +export type ProjectLinkKey = keyof typeof PROJECT_LINKS + +/** Translation key of each project link label, shared by the drawer and the footer. */ +export const PROJECT_LINK_LABEL_KEYS: Record = { + github: "nav_github", + npm: "footer_link_npm", + docs: "footer_link_docs", + apiReference: "footer_link_api", + benchmarks: "footer_link_benchmarks", + releases: "footer_link_releases", + changelog: "footer_link_changelog", + license: "footer_link_license", + commercialLicense: "footer_link_commercial_license", +} + +/** The footer order: where the code lives first, then documentation, then licensing. */ +export const FOOTER_LINK_KEYS: readonly ProjectLinkKey[] = [ + "github", + "npm", + "docs", + "apiReference", + "benchmarks", + "releases", + "changelog", + "license", + "commercialLicense", +] diff --git a/packages/ragmir-landing/src/pages/[...locale]/index.astro b/packages/ragmir-landing/src/pages/[...locale]/index.astro index 1aa8117..6cb868d 100644 --- a/packages/ragmir-landing/src/pages/[...locale]/index.astro +++ b/packages/ragmir-landing/src/pages/[...locale]/index.astro @@ -14,6 +14,7 @@ import Strengths from "../../components/sections/strengths.astro" import UseCases from "../../components/sections/use-cases.astro" import { defaultLang, getLocalizedUrl, languages, locales, useTranslations } from "../../i18n/utils" import Layout from "../../layouts/layout.astro" +import { PROJECT_LINKS } from "../../lib/project-links" export function getStaticPaths() { return locales.map((locale) => ({ @@ -76,22 +77,21 @@ const structuredData = [ abstract: sourceDescription, url: siteUrl, inLanguage: ["en", "fr"], - codeRepository: "https://github.com/jcode-works/jcode-ragmir", + codeRepository: PROJECT_LINKS.github, codeSampleType: "full solution", runtimePlatform: "Node.js 22 on Linux x64 or macOS ARM64", programmingLanguage: "TypeScript", keywords: ["agentic RAG", "local RAG", "coding agents", "cited retrieval", "MCP", "TypeScript", "AGPL open source"], - license: "https://github.com/jcode-works/jcode-ragmir/blob/main/LICENSE", - usageInfo: - "https://github.com/jcode-works/jcode-ragmir/blob/main/COMMERCIAL-LICENSE.md", + license: PROJECT_LINKS.license, + usageInfo: PROJECT_LINKS.commercialLicense, isAccessibleForFree: true, ...(publicVersion ? { version: publicVersion } : {}), isPartOf: { "@id": "https://ragmir.com/#website", }, sameAs: [ - "https://github.com/jcode-works/jcode-ragmir", - "https://www.npmjs.com/package/@jcode.labs/ragmir", + PROJECT_LINKS.github, + PROJECT_LINKS.npm, ], publisher: { "@id": "https://ragmir.com/#organization", @@ -121,6 +121,7 @@ const localizedLibraryUrl = `${getLocalizedUrl("/", locale)}#library` const localizedFeaturesUrl = `${getLocalizedUrl("/", locale)}#features` const localizedUseCasesUrl = `${getLocalizedUrl("/", locale)}#use-cases` const localizedAgentsUrl = `${getLocalizedUrl("/", locale)}#agents` +const localizedFaqUrl = `${getLocalizedUrl("/", locale)}#faq` const localizedTeamUrl = getLocalizedUrl("/team", locale) --- @@ -139,7 +140,9 @@ const localizedTeamUrl = getLocalizedUrl("/team", locale) client:load alternateLocales={alternateLocales} locale={locale} + currentPath={Astro.url.pathname} localizedAgentsUrl={localizedAgentsUrl} + localizedFaqUrl={localizedFaqUrl} localizedFeaturesUrl={localizedFeaturesUrl} localizedHomeUrl={localizedHomeUrl} localizedLibraryUrl={localizedLibraryUrl} diff --git a/packages/ragmir-landing/src/pages/[...locale]/team.astro b/packages/ragmir-landing/src/pages/[...locale]/team.astro index 28a1bbe..72a943a 100644 --- a/packages/ragmir-landing/src/pages/[...locale]/team.astro +++ b/packages/ragmir-landing/src/pages/[...locale]/team.astro @@ -92,7 +92,9 @@ const structuredData = [ client:load alternateLocales={alternateLocales} locale={locale} + currentPath={Astro.url.pathname} localizedAgentsUrl={`${homeUrl}#agents`} + localizedFaqUrl={`${homeUrl}#faq`} localizedFeaturesUrl={`${homeUrl}#features`} localizedHomeUrl={`${homeUrl}#top`} localizedLibraryUrl={`${homeUrl}#library`} diff --git a/packages/ragmir-landing/src/public-contract.test.ts b/packages/ragmir-landing/src/public-contract.test.ts index e853abf..1086c2c 100644 --- a/packages/ragmir-landing/src/public-contract.test.ts +++ b/packages/ragmir-landing/src/public-contract.test.ts @@ -9,6 +9,7 @@ import { USE_CASES } from "./components/use-case-carousel.js" import { getFaqItems } from "./content/faq.js" import { RAGMIR_SETUP_PROMPT } from "./content/setup-prompt.js" import { getLocalizedUrl, loadTranslations, useTranslations } from "./i18n/utils.js" +import { FOOTER_LINK_KEYS, PROJECT_LINK_LABEL_KEYS, PROJECT_LINKS } from "./lib/project-links.js" import { cn } from "./lib/utils.js" const homePageSource = readFileSync( @@ -64,6 +65,10 @@ const footerSource = readFileSync( fileURLToPath(new URL("./components/sections/footer.astro", import.meta.url)), "utf8", ) +const navbarSource = readFileSync( + fileURLToPath(new URL("./components/landing-navbar.tsx", import.meta.url)), + "utf8", +) const textareaSource = readFileSync( fileURLToPath(new URL("./components/ui/textarea.tsx", import.meta.url)), "utf8", @@ -126,11 +131,37 @@ describe("landing public contract", () => { }) it("should back the hero with the measured benchmark numbers of the README", () => { - expect([en.hero_proof_recall_value, en.hero_proof_speed_value]).toEqual(["93%", "40 ms"]) - expect([fr.hero_proof_recall_value, fr.hero_proof_speed_value]).toEqual(["93 %", "40 ms"]) - expect(en.hero_proof_speed_label).toContain("43,874-chunk monorepo") + expect([en.hero_proof_recall_value, en.hero_proof_speed_value]).toEqual(["93%", "39 ms"]) + expect([fr.hero_proof_recall_value, fr.hero_proof_speed_value]).toEqual(["93 %", "39 ms"]) + expect(en.hero_proof_speed_label).toContain("29,778-chunk monorepo") expect(heroSource).toContain('t("hero_proof_recall_value")') - expect(heroSource).toContain("https://github.com/jcode-works/jcode-ragmir#benchmarks") + expect(PROJECT_LINKS.benchmarks).toBe("https://github.com/jcode-works/jcode-ragmir#benchmarks") + expect(heroSource).toContain("PROJECT_LINKS.benchmarks") + }) + + it("should send visitors to GitHub and npm from the hero and the mobile drawer", () => { + expect(PROJECT_LINKS.github).toBe("https://github.com/jcode-works/jcode-ragmir") + expect(PROJECT_LINKS.npm).toBe("https://www.npmjs.com/package/@jcode.labs/ragmir") + for (const source of [heroSource, navbarSource]) { + expect(source).toContain('t("hero_github_cta")') + expect(source).toContain('t("hero_npm_cta")') + } + expect([en.hero_github_cta, en.hero_npm_cta]).toEqual(["View on GitHub", "View on npm"]) + expect([fr.hero_github_cta, fr.hero_npm_cta]).toEqual(["Voir sur GitHub", "Voir sur npm"]) + }) + + it("should list every project link in the footer and label it in both locales", () => { + expect([...FOOTER_LINK_KEYS].sort()).toEqual(Object.keys(PROJECT_LINKS).sort()) + expect(footerSource).toContain("FOOTER_LINK_KEYS.map") + expect(navbarSource).toContain("PROJECT_LINKS[key]") + for (const key of Object.values(PROJECT_LINK_LABEL_KEYS)) { + expect(en).toHaveProperty(key) + expect(fr).toHaveProperty(key) + } + for (const key of ["nav_faq", "nav_section_docs", "nav_section_project"]) { + expect(en).toHaveProperty(key) + expect(fr).toHaveProperty(key) + } }) it("should position Ragmir as the evidence layer for agentic RAG workflows", () => { @@ -225,12 +256,14 @@ describe("landing public contract", () => { expect(en.footer_link_commercial_license).toBe("Commercial license") expect(fr.footer_link_commercial_license).toBe("Licence commerciale") expect(closingCtaSource).toContain('t("closing_open_source")') - expect(footerSource).toContain("COMMERCIAL-LICENSE.md") - expect(homePageSource).toContain( - 'license: "https://github.com/jcode-works/jcode-ragmir/blob/main/LICENSE"', + expect(PROJECT_LINKS.commercialLicense).toContain("COMMERCIAL-LICENSE.md") + expect(PROJECT_LINKS.license).toBe( + "https://github.com/jcode-works/jcode-ragmir/blob/main/LICENSE", ) - expect(homePageSource).toContain("usageInfo:") - expect(homePageSource).toContain("COMMERCIAL-LICENSE.md") + expect(FOOTER_LINK_KEYS).toContain("commercialLicense") + expect(FOOTER_LINK_KEYS).toContain("license") + expect(homePageSource).toContain("license: PROJECT_LINKS.license") + expect(homePageSource).toContain("usageInfo: PROJECT_LINKS.commercialLicense") }) it("should provide every localized key referenced by hero scenarios", () => { From 858663c437050caa4ea46920963677b588272435 Mon Sep 17 00:00:00 2001 From: Jean-Baptiste THERY Date: Sun, 1 Nov 2026 19:17:54 +0700 Subject: [PATCH 5/5] docs: publish the measured abstention gains and the stack comparison - lead both READMEs with the current measured recall and latency of the monorepo index - show Ragmir 6.4 against 6.3.1 on three real bases: same answers, more refusals of off-topic and near-domain questions, fewer correct answers flagged as weak - add the comparison with LangChain.js, Orama, and MiniSearch on this repository --- README.md | 79 +++++++++++++++++++++++----------- packages/ragmir-core/README.md | 10 +++-- 2 files changed, 61 insertions(+), 28 deletions(-) diff --git a/README.md b/README.md index 0267d07..65fb0b7 100644 --- a/README.md +++ b/README.md @@ -11,8 +11,8 @@

- 93% of real developer questions find the right file in the top 10 · 40 ms median search on a - 43,874-chunk monorepo · benchmarks + 93% of real developer questions find the right file in the top 10 · 39 ms median search on a + 29,778-chunk monorepo · benchmarks

@@ -56,12 +56,18 @@ npx rgr search "how are permissions checked?" --compact - **Code and documents in one index.** Hybrid BM25 and vector search understands paths and camelCase identifiers and boosts the code that defines what you ask about, so "how are permissions checked" reaches `PermissionService.java`, not only its tests and notes. -- **Fast on real repositories.** Median search takes 40 ms on a 43,874-chunk Java and TypeScript +- **Fast on real repositories.** Median search takes 39 ms on a 29,778-chunk Java and TypeScript monorepo with Office files, and the right file reaches the top 10 for 93% of 101 real developer - questions (87% with 6.1.6). + questions. +- **Ahead of the usual RAG stacks.** On this repository, the right file reaches the top 3 for 76% + of developer questions, against 56% for a LangChain.js pipeline and 70% for MiniSearch BM25 with + the same embedding model, and Ragmir refuses 27 of 30 off-topic questions where they refuse at + most 3. - **Evidence you can verify.** Citations point to source lines, PDF pages, slides, sheet cells, or EPUB sections. Results say when their file changed after indexing (`sourceStatus`) and when - their support is thin (`weakEvidence`); unrelated questions return nothing instead of noise. + their support is thin (`weakEvidence`). Questions your sources cannot answer return nothing + instead of noise: 58 of 90 off-topic and 24 of 90 near-domain questions on real bases, up from + 41 and 1. - **Always current.** `rgr doctor` compares the files on disk with the index on every run, and `rgr hooks install` refreshes it in the background after commits, merges, and checkouts. - **Private by default.** No account, telemetry, or hosted storage. The default `local-hash` @@ -180,25 +186,50 @@ use the same retrieval API without one. ## Benchmarks -Ragmir 6.2 against 6.1.6 on the same machine, configuration, and index, rebuilt by 6.2. Each -answerable question has one expected file; "top 3" and "top 10" count the questions whose file -appears there. Unrelated questions (nonsense, weather, recipes, sports) should return nothing. -Latency is the median of warm sequential searches in one process. - -| Corpus | Embeddings | Answer in top 3 | Answer in top 10 | nDCG@10 | Unrelated questions refused | Median search | -| --- | --- | ---: | ---: | ---: | ---: | ---: | -| Java/TypeScript monorepo with Office files and scans: 2,117 files, 43,874 chunks, 101 developer questions | multilingual-e5-small | 70% → **73%** | 87% → **93%** | 0.639 → **0.670** | 0 → **3** of 30 | 31 → 40 ms | -| French and English product documentation: 298 files, 20,931 chunks, 60 questions | mxbai-embed-xsmall | 100% → **100%** | 100% → **100%** | 0.932 → **0.938** | 3 → **15** of 30 | 25 → 31 ms | - -Measured on October 27, 2026 on an Apple M3 Max with Node.js 22.23 and LanceDB 0.30, with the -`quality` profile these projects use and 30 unrelated questions per corpus. The two private corpora -are described rather than published. Of the passages 6.2 still returned for unrelated questions, -it flagged 41 of 81 on the monorepo and 37 of 38 on the documentation as weak evidence, against -7 of 94 and 0 of 60 correct answers. On the monorepo it also applied six exclusion rules that -6.1.6 silently missed on accented file names and cited 27 decomposed paths in canonical form. The -extra milliseconds pay for word statistics, code-definition boosts, and crowding checks. Run the -[benchmarks](./packages/ragmir-core/benchmarks/README.md) and `rgr evaluate` on your own corpus -before relying on any number. +### Real projects + +Ragmir 6.4 against 6.3.1 on the same machine, configuration, and indexes. Each answerable question +lists the files that answer it; "top 3" and "top 10" count the questions with one of them there. +Off-topic questions (sports, cooking, weather, nonsense) and near-domain questions about features +the project does not have should return nothing. Latency is the median of warm sequential searches +in one process. + +| Corpus | Embeddings | Answer in top 3 | Answer in top 10 | Off-topic refused | Near-domain refused | Correct answers flagged weak | Median search | +| --- | --- | ---: | ---: | ---: | ---: | ---: | ---: | +| Java/TypeScript monorepo with Office files and scans: 2,297 files, 29,778 chunks, 101 developer questions | multilingual-e5-base | 74% → **74%** | 93% → **93%** | 0 → **16** of 30 | 1 → **11** of 30 | 11 → **8** of 94 | 39 → 39 ms | +| French and English product documentation: 299 files, 21,011 chunks, 60 questions | mxbai-embed-xsmall | 100% → **100%** | 100% → **100%** | 16 → **17** of 30 | 0 → **6** of 30 | 0 → **0** of 60 | 28 → 28 ms | +| This repository: 229 files, 2,490 chunks, 99 developer questions | mxbai-embed-xsmall | 75% → **75%** | 97% → **97%** | 25 → **25** of 30 | 0 → **7** of 30 | 32 → **6** of 96 | 22 → 22 ms | + +Measured on November 1, 2026 on an Apple M3 Max with Node.js 26 and LanceDB 0.30, with the +`quality` profile these projects use. Near-domain questions pair the project's own vocabulary with +an integration, service, or format it does not have, and each was checked absent from every +indexed file. 6.4 kept every answer in place and caught them by measuring each index's noise +floor: a passage only counts as a strong vector match when it is nearer than 24 off-topic probe +questions, and a search whose first five results are all weak answers nothing. The two private +corpora are described rather than published. Run the [benchmarks](./packages/ragmir-core/benchmarks/README.md) +and `rgr evaluate` on your own corpus before relying on any number. + +### Against other retrieval stacks + +`pnpm bench:competitors` indexes the 210 tracked documentation and source files of this +repository and asks its 99 developer questions plus 30 off-topic and 30 near-domain questions. Every +vector side uses the same model and vectors (`mxbai-embed-xsmall-v1`); the other stacks read the same +files split by LangChain's `RecursiveCharacterTextSplitter` (1,000 characters, 200 overlap). + +| System | Answer in top 3 | Answer in top 10 | MRR@10 | Off-topic refused | Near-domain refused | Median search | +| --- | ---: | ---: | ---: | ---: | ---: | ---: | +| **Ragmir**, hybrid with `mxbai-embed-xsmall-v1` | **76%** | **97%** | **0.647** | **27** of 30 | **5** of 30 | 22 ms | +| **Ragmir**, `local-hash` without a model | **79%** | 96% | 0.611 | 9 of 30 | 0 of 30 | 20 ms | +| LangChain.js splitter with exact cosine search, as `MemoryVectorStore` | 56% | 89% | 0.509 | 0 of 30 | 0 of 30 | 5 ms | +| Orama 3 hybrid, default similarity 0.8 | 18% | 35% | 0.144 | 2 of 30 | 0 of 30 | 5 ms | +| Orama 3 hybrid, similarity 0.1 | 38% | 64% | 0.317 | 0 of 30 | 0 of 30 | 6 ms | +| MiniSearch 7 BM25 | 70% | 92% | 0.566 | 3 of 30 | 0 of 30 | 1 ms | + +In-memory libraries answer faster, but they return results for almost every off-topic question, +and none ranks the answering file as high: it reaches the top 3 for 76 to 79% of questions with Ragmir, +against 56% with the LangChain.js pipeline and 70% with MiniSearch. Measured on the same machine on +November 1, 2026; [the benchmark guide](./packages/ragmir-core/benchmarks/README.md) explains every +setting. ## TypeScript diff --git a/packages/ragmir-core/README.md b/packages/ragmir-core/README.md index 3b4f484..81fe5e0 100644 --- a/packages/ragmir-core/README.md +++ b/packages/ragmir-core/README.md @@ -11,8 +11,8 @@

- 93% of real developer questions find the right file in the top 10 · 40 ms median search on a - 43,874-chunk monorepo · benchmarks + 93% of real developer questions find the right file in the top 10 · 39 ms median search on a + 29,778-chunk monorepo · benchmarks

@@ -31,12 +31,14 @@ model you choose. Ragmir does not bundle a chat model or host your documents. - **Code and documents in one index:** hybrid BM25 and vector search that understands paths and camelCase identifiers and boosts the code that defines what you ask about. -- **Fast on real repositories:** 40 ms median search on a 43,874-chunk monorepo, with the right +- **Fast on real repositories:** 39 ms median search on a 29,778-chunk monorepo, with the right source in the top 10 for 93% of real developer questions ([benchmarks](https://github.com/jcode-works/jcode-ragmir#benchmarks)). +- **Ahead of the usual RAG stacks:** the right file reaches the top 3 for 76% of this repository's + developer questions, against 56% for a LangChain.js pipeline and 70% for MiniSearch BM25. - **Evidence you can verify:** citations point to source lines or native page, slide, sheet/cell, and EPUB coordinates. Results flag changed files (`sourceStatus`) and thin support - (`weakEvidence`); unrelated questions return nothing. + (`weakEvidence`); questions your sources cannot answer return nothing instead of noise. - **Always current:** `rgr doctor` checks the files on disk on every run, and `rgr hooks install` refreshes the index after commits, merges, and checkouts. - **Private and portable:** offline by default, with no account or telemetry, and the same paths,